mobiperf / mobiperf Goto Github PK

New MobiPerf Repository

License: Other

Java 62.84% Python 33.22% Shell 1.18% CSS 2.77%

mobiperf's Introduction

This is a project to develop a mobile app and service for collecting
mobile network measurements. This is intended for experimental use
only.

Home page:
  https://github.com/Mobiperf/Mobiperf

This code is released under the Apache 2.0 License (see LICENSE for
details).

How to Contribute to this Codebase
----------------------------------

This code can be found here:
  https://github.com/Mobiperf/Mobiperf

If you wish to contribute, please read the instructions in

  docs/code-review.txt

Note that simply issuing pull requests to the Mobiperf repository
on github will not suffice.

Overview of this code
---------------------

Mobiperf consists of an Android app that collects mobile network
measurements, and a data collection server, which runs on AppEngine.

The Android app periodically checks in with the measurement server,
which sends it a list of measurement tasks to perform. Measurement tasks
include ping, traceroute, HTTP GET, DNS lookup, TCP Throughput, IPv4/v6 
compatability check and UDP Burst. Each task has an associated set of 
measurement parameters (e.g., which host to ping), and a schedule
(periodicity at which to take the measurement).

The device runs the measurements in the background, and uploads the
measurement results on its next checkin cycle. By default, devices
check in with the server every hour. In order to avoid draining the
battery, the app will not take any measurements if the battery is
below a given threshold (80% by default).

The protocol spoken between the app and the server is a simple JSON
REST API, documented below.

The server is implemented using Google AppEngine. It is responsible
for managing device checkins, handing out measurement tasks, and
collecting results. It has a simple Web dashboard for presenting
measurement results to the user. There is also a mechanism for
querying measurement results.

Protocol Description
--------------------

The Mobiperf app communicates with the Mobiperf service using
the following protocol, using JSON-encoded payloads over HTTP POST
methods.

All strings representing dates and times are represented in ISO8601
format with a trailing Zindicating UTC, e.g.,
  2011-06-22T10:27:24.204589Z

/checkin - Used by devices to periodically check in with the service.

Input:

{
 "id": "device ID",
 /* Note that the user field is implied */
 "manufacturer": "device manufacturer",
 "model": "device model",
 "os": "device OS",
 "properties": {
    "timestamp": "timestamp in ISO8601 format",
    "os_version": "device OS version",
    "location": {
      "latitude": latitude value as floating point decimal degrees,
      "longitude": longitude value as floating point decimal degrees
    }
    "location_type": "location type string",
    "network_type": "network type string",
    "carrier": "carrier identifier string",
  }
}

Output: A list of JSON objects representing task descriptors for the
measurements that the device should perform. This schedule overwrites
any existing schedule on the device. Ongoing measurement tasks not
included in the new schedule are terminated immediately. If the
schedule is empty, the device will stop performing measurements.

[
  { "key": "optional task key",
    "type": "measurement type",
    "parameters": [
      { "paramName1": "paramValue1" },
      { "paramName2": "paramValue2" },
      ...
    ]
    "start_time": "task start time in ISO8601 format",
    "end_time": "task end time in ISO8601 format",
    "interval_sec": interval as an integer,
    "count": count as an integer,
    "priority": priority as an integer
  },
  { /* Additional Task... */ },
  { /* Additional Task... */ }
]

/postmeasurement - Used by the device to post a set of measurement
results to the service.

Input: JSON representation of a list of Measurement objects, with
embedded DeviceProperties.

  [ { "device_id": "device ID",
      "properties": {
      "timestamp": "timestamp in ISO8601 format",
      "os_version": "device OS version",
      "location": {
        "latitude": latitude value as floating point decimal degrees,
        "longitude": longitude value as floating point decimal degrees
       }
      "location_type": "location type string",
      "network_type": "network type string",
      "carrier": "carrier identifier string",
    }
    "type": "measurement type",
    "timestamp": "measurement timestamp in ISO8601 format",
    "success": true or false,
    "task_key": "task key associated with this measurement, if any",
    "parameters": [
        { "paramName1": "paramValue1" },
        { "paramName2": "paramValue2" },
        ...
    ],
    "values": [
        { "valueName1": "valueValue1" },
        { "valueName2": "valueValue2" },
        ...
    ],
  },
  { /* Additional Measurement... */ },
  { /* Additional Measurement... */ },
]

Output: A JSON-encoded representation of whether the post was
successful:

  { "success": true or false }

The measurement type may be "ping", "traceroute", "dns", or "http".
The parameters and measurement values for each measurement type are
given below. All parameters and values are strings (since this is how
they are represented in the JSON protocol described above).

Ping parameters

(required) target - the host name or IP address of the server to ping
(optional) packet_size_byte - the packet per ICMP ping in the unit of
bytes. Default to 56.
(optional) ping_timeout_sec - the number of seconds we wait for a ping
response. Default to 0.5.

Ping values

(required) target_ip - the IP address of the target we ping against
(required) ping_method - the actual ping method this result represents.
(ping_cmd, java_ping, http).
(required) mean_rtt_ms - Mean RTT in milliseconds (e.g.,
 "20.485" for 20.485 ms)
(required) min_rtt_ms - Min RTT in milliseconds
(required) max_rtt_ms - Max RTT in milliseconds
(required) stddev_rtt_ms - Standard deviation of RTT in milliseconds
(optional) filtered_mean_rrt_ms - Mean RRT with outlier values filtered
out. Unit is in milliseconds
(optional) packet_loss - Fraction of lost packets (e.g., "0.5" for 50% loss)

Traceroute parameters

(required) target - the hostname or IP address to use as the target of
the traceroute.
(required) packet_size_byte - the packet per ICMP ping in the unit of
bytes. Default to 56.
(optional)  ping_timeout_sec - the number of seconds we wait for a ping
response. Default to 2.
optinal ping_interval_sec - the interval between successive pings in
seconds. Default to 0.5.
optinal pings_per_hop - the number of pings we use for each ttl value.
Default to 3.
(optional) max_hop_count - the total number of hops we ping before we
declare the traceroute fails. Default to 10.

Traceroute results

(required) num_hops - Number of observed hops in the traceroute
(required) hop_N_addr_i - The ith IP address of the Nth hop along the
observed route, where N ranges from 0 to num_hops-1.
(required) hop_N_rtt_ms - Observed RTT in milliseconds to this hop.

DNS lookup parameters

(required) target - Hostname of the target to resolve
(optional) server - IP address of a DNS server to use as the resolver.
If not present, the device's default resolver is used.

DNS lookup values

(required) address - IPv4 address of the target as returned by an A
record
(required) real_hostname - True FQDN of the host that has been resolved
(required) time_ms - Time taken to perform the DNS lookup

HTTP parameters

(required) url - URL to request
(optional) method - HTTP method to use. Defaults to "GET"
(optional) headers - String (possibly containing newlines) with
additional headers to send with the request. Each header and value
pair is in the form of "headerParam:value", with different
pairs separated by "\r\n".
(optional) body - String with the request body to send (if method is
"POST")

HTTP values

(required) time_ms - Time in milliseconds to perform the complete
request
(required) code - Response code (e.g., "200")
(optional) headers_len - Size in bytes of the original response headers
(optional) body_len - Size in bytes of the original response body
(optional) headers - Response headers - may be compressed, truncated, or
elided in the case of a large response
(optional) body - Response body - may be compressed, truncated, or
elided in the case of a large response. It is a JSON encoded byte
array.

TCP Throughput parameters

(required) dir_up - Uplink or Downlink measurement (boolean)
(required) target - hostname for servers. Use m-lab for now. (string)
(optional) data_limit_mb_up - Uplink cellular network data 
limit (double)
(optional) data_limit_mb_down - Downlink cellular network data 
limit (double)
(optional) duration_period_sec - Downlink maximum experiment 
duration period (double)
(optional) pkt_size_up_bytes - The size each packet in the uplink (int)
(optional) sample_period_sec - The small interval to calculate current 
throughput result (double)
(optional) slow_start_period_sec - Waiting period to avoid TCP slow 
start (double)
(optional) tcp_timeout_sec - TCP connection timeout (double)

TCP Throughput value

(required) tcp_speed_results - A list of throughput sampling results 
in kbps (list of double)
(required) data_limit_exceeded - A flag indicating transmitted data 
exceeding limit (boolean)
(required) duration - Time to finish the task (double)
(required) server_version - M-Lab server side code version (string)

mobiperf's People

Contributors

Stargazers

Watchers

mobiperf's Issues

App crash

Bug found.

I've run MobiPerf this morning. I'm running it with Android 2.3.3, with WiFi access.

Attached is the debug.txt (output of adb logcat *:s 'Mobiperf') and bugreport.txt (output of adb bugreport).

What I've done is, first run an UDP down experiment (the result says PRR 0.0, so not sucessful?), then run an UDP up experiment (the results says experiment has failed, I'm not sure why). Then I go to results tab and also looked at the system logs. Then the app crashes.

I'm not exactly sure the cause of the crash, but I think it is NOT caused by this exception "java.lang.RuntimeException: No google account found" since I've observed the app working fine while producing it.

The file attachments are in the email. Search for "debug.txt" or "bugreport.txt" in your emails.

AppEngine support for anonymous data collection

Allow data to be collected even if user is not logged in.

Setting account doesn't store auth cookie

I noticed the scheduled tests were failing when anonymous so I set an account. This worked until i restarted, now i have no auth cookie again and tests are failing. Settings do show that an account is selected.

Bug: System log empty

The system log needs to show which actions the app took, which we can use for debugging. This needs to persist across app invocations. When I start up the app the system log is empty.

Notification at the top indicating whether MobiPerf is running in the background

Add notification at the top indicating whether MobiPerf is running in the background, i.e., whether periodic setting is enabled or not in the preference page

Add a Google map with aggregate statistics on data that has been collected

Sproadic crashes at start

The app still occasionally crashes at startup, as well as when viewing the console screen. Likely due to how we are sharing data between the MeasurementScheduler and UI tasks.

adb bugreport shows:

8-30 11:32:54.294 25342 25342 E AndroidRuntime: FATAL EXCEPTION: main
08-30 11:32:54.294 25342 25342 E AndroidRuntime: java.lang.IllegalStateException: The content of the adapter has changed but ListView did not receive a notification. Make sure the content of your adapter is not modified from a background thread, but only from the UI thread. [in ListView(2131361792, class android.widget.ListView) with Adapter(class android.widget.ArrayAdapter)]
08-30 11:32:54.294 25342 25342 E AndroidRuntime: at android.widget.ListView.layoutChildren(ListView.java:1538)
08-30 11:32:54.294 25342 25342 E AndroidRuntime: at android.widget.AbsListView$CheckForTap.run(AbsListView.java:3069)
08-30 11:32:54.294 25342 25342 E AndroidRuntime: at android.os.Handler.handleCallback(Handler.java:615)
08-30 11:32:54.294 25342 25342 E AndroidRuntime: at android.os.Handler.dispatchMessage(Handler.java:92)
08-30 11:32:54.294 25342 25342 E AndroidRuntime: at android.os.Looper.loop(Looper.java:137)
08-30 11:32:54.294 25342 25342 E AndroidRuntime: at android.app.ActivityThread.main(ActivityThread.java:4745)
08-30 11:32:54.294 25342 25342 E AndroidRuntime: at java.lang.reflect.Method.invokeNative(Native Method)
08-30 11:32:54.294 25342 25342 E AndroidRuntime: at java.lang.reflect.Method.invoke(Method.java:511)
08-30 11:32:54.294 25342 25342 E AndroidRuntime: at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:786)
08-30 11:32:54.294 25342 25342 E AndroidRuntime: at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:553)
08-30 11:32:54.294 25342 25342 E AndroidRuntime: at dalvik.system.NativeStart.main(Native Method)
08-30 11:32:54.309 291 2088 W ActivityManager: Force finishing activity com.mobiperf.speedometer/.SystemConsoleActivity

Feature: Rename periodic to background

"Periodic running" should be renamed to "Background measurements".

This should be ON by default.

Make MobiPerf automatically starts when reboot

Check whether MobiPerf automatically starts when reboot, if not, implement it

Update readme for new tests (TP/RTT)

Update readme for new tests (TP/RTT) including the detailed protocol used, parameters.
Also update the privacy doc in Google Doc about these new tests.

Traceroute result presentation error

One problem I found is about the traceroute measurement.

The result shown in the Results tab => Manual has some problems.

For the first hop:
"1 192.168.1. 37.3ms" it should be => "1 192.168.1.110 37.3ms"
Among the 15 hops in my case, 3 hops have this problem, that the last number of the IP address is missing.
But the other 12 hops are fine.
One suggestion I have is, to align the RTT numbers, e.g., 37.3ms in the example, to the right, while align the IP to the left.

How to implement agreement and selected account?

Currently, when MobiPerf is first run, user will see an agreement asking to agree or just quit.
If user clicks agree, the current MobiPerf would write something into a file in MobiPerf's sandbox.
Next time, just check this file. This method works for the entire life time of an app, even when the device is rebooted, users don't need to click agree again.

Another way to implement, is to use a static variable and change the values of it.
This way, when the app is killed/forced stopeed or the device is rebooted, users will see the agreement window again.

Which of the above two ways is better for our app?

Similar problem exists for selecting account for checkin, i.e., where the selected account should be stored? File or static variable?

Add status notification in the Android system notification area at top

Add status notification in the Android system notification area at top.
When the app is started when the phone reboots
When the app is running in the background

Feature: "About Us" page needs to be filled in

The "About Us" Page in the app needs real content.

Report GPS location before and after each measurement

Will be useful for estimating whether the device was in motion during the measurement.
Would require changes to protocol and measurement schema to record > 1 location per measurement.

Crash on startup in emulator

I set up an emulator on android-8 (level 12) to test the min supported version and I have a 100% crash on startup:

09-24 15:28:46.561 I/am_crash( 65): [318,com.mobiperf.speedometer,48710,java.lang.NullPointerException,Unable to start activity ComponentInfo{com.mobiperf.speedometer/com.mobiperf.speedometer.SpeedometerApp}: java.lang.NullPointerException,AccountSelector.java,118]

Measurement results not persisted across scheduler restarts

Measurement results are not persisted across scheduler restarts, meaning that measurements can be taken but lost (due to OOM eviction) before they are uploaded.

Options to clear all displayed results/server collected results

Give users the options to clear all displayed results, including manual, automatic results and system logs.

(optional) give users the option to clear all server collected results from GAE.

gpsEnabled is set to false by default.

It should be true.
Also, even after setting through preferences it's reset after rebooting (or process killed and rerun) as the value is not loaded from preferences.

Bug: Results tabs have two lines that say results will appear

Both the manual and scheduled results tabs have TWO lines that say results will appear there. Initially there should be only one, and it should go away once results have been posted.

Crashing bug for MobiPerf

When you first open MobiPerf, then go to System's settings => applications => MobiPerf, force stop => reopen MobiPerf, mobiperf will crash.

The reason is because PhoneUtils's setGlobalContext is not called before the getPhoneUtils in a rare case.

Boot Receiver bug

The manifest file has the wrong package name specified for the boot receiver:

com.mobiperf.mobiperf.WatchdogBootReceiver

should be

com.speedometer.speed.WatchdogBootReceiver

... or you can move the class to the package specified in the manifest.

H/t to Seungyeop for finding the root cause.

System log shows measurement "running" when in fact phone is on WiFi

Should not say "Running" in the system log when the measurement is being skipped due to WiFi or battery threshold.

Sporadic crashes at start

There still seems to be a race condition when the app starts which causes an app crash.

I also noticed that the app would hang when sitting on the System Console screen possibly while the console was updating - likely race condition there. There are probably related to how we are sharing state between the MeasurementScheduler and UI activities.

adb bugreport shows:

Device ID selection issue

Take a look at http://stackoverflow.com/questions/2785485/is-there-a-unique-android-device-id

What device ID to use?

Android ID has some problem, some devices have duplicated Android IDs?

TelephonyManager.getDeviceID()? sometimes not available

Get a unique ID from server? can't correlate the same user.

Need to discuss later

README is lacking

I had no idea that there was documentation. My understanding is that functionally very little has changed from the speedometer code. With that in mind can we update the MobiPerf README with the stuff from here (with changes as needed of course).

https://github.com/Mobiperf/MobiPerf/blob/master/README

Version numbering needs to be cleaned up

Version numbers are a mess. The app splash screen and "About" page say 1.0. The Android Settings > Apps page for Mobiperf says 0.2. There needs to be a consistent version numbering scheme and this needs to be very clear to the user.

Add opt-out option for location sharing

Add opt-out option for location sharing.
Add a check box in the settings page.
If user specifies not to share their location, a mock location is used.

Need an in-app bug reporting mechanism

For dogfood/beta release, simply having a clear email link in the app to report a bug (e.g., to [email protected]) is fine.

Longer term, we should consider embedding a bug reporting library, e.g.,

http://www.hockeyapp.net/
http://code.google.com/p/acra/

Improved security for device ID

Change IMEI to contain the TAC + a hashed/salted version of the rest of the IMEI

Traceroute problem

When I use MobiPerf to traceroute to www.ebay.com, it says failure.

However, when I traceroute in my computer, I can see that the reason is that from hop 11, there are all timeouts. So I think the timeout/error handling of the existing traceroute test module can be improved

traceroute www.ebay.com
traceroute: Warning: www.ebay.com has multiple addresses; using 66.211.181.161
traceroute to www.ebay.com (66.211.181.161), 64 hops max, 52 byte packets
1 141.212.110.1 (141.212.110.1) 0.834 ms 0.581 ms 0.586 ms
2 vss-cse.engin.umich.edu (141.213.127.134) 1.282 ms 0.519 ms 0.360 ms
3 l3-caen-bin-arb.r-bin-arbl.umnet.umich.edu (192.12.80.177) 0.397 ms 0.389 ms 0.418 ms
4 l3-barb-bseb-2.r-bin-seb.umnet.umich.edu (192.12.80.11) 0.683 ms 0.688 ms 0.603 ms
5 v-bin-seb-inet-aa2.aa2.mich.net (192.12.80.37) 0.777 ms 0.637 ms 0.620 ms
6 xe-5-1-0x76.eq-chi2.mich.net (198.108.23.12) 6.512 ms 6.540 ms 6.512 ms
7 12.250.16.17 (12.250.16.17) 15.162 ms 32.018 ms 49.174 ms
8 cr1.cgcil.ip.att.net (12.122.133.122) 32.422 ms 31.161 ms 31.920 ms
9 cr2.dvmco.ip.att.net (12.122.31.85) 30.657 ms 29.535 ms 32.959 ms
10 12.123.38.141 (12.123.38.141) 27.730 ms 27.804 ms 27.773 ms
11 * * *
12 * * *
13 * * *
14 * * *

Version number needs to be meaningful

The OLD (and very broken) APK I have on my phone reports itself as being "v1.0". This is not v1.0, far from it. Please revert the version number to something meaningful so we can talk about which version we are on. I propose 0.2 since what I am running now is more like 0.1.

Need to finalize anonymization pipeline

Show steps of how to upload MobiPerf server from windows PC

Splash screen size not matching

Current splash screen is a little "short" and there are some white boundaries.
Maybe in the future we can first detect the screen size and make the splash screen stretch a little bit to fit into the whole screen?

Change "about" link on openmobiledata to point to mobiperf.com

Remove all PII from RTT and Throughput test and add to current MobiPerf

Remove all PII from RTT and Throughput test and add to current MobiPerf.

Replace IMEI with a 128-bit UUID (generated for each test, not for each device), which could be used to link Android data with MLab tcpdumps.

Send the UUID to MLab and GAE using JSON protocol.

Settings to select account again/ sign out

Add settings to allow the user to change their authentication account anytime (not just the first time they run the app) and allow them to sign out.

Whether to allow them to sign out is to be discussed.
If we do, do we allow anonymous experiments?
If not, we can't allow them to sign out.

Bug: Scheduled results not persisting across runs

I have been running the MobiPerf app and much of the time, when I start it up the "Scheduled measurements" tab under "Results" is empty (well, it contains TWO lines that say that the measurements will appear there). Whatever logic is supposed to be persisting the results across app activations is not working.

Add MobiPerfUmich tests to MobiPerf

Add uplink/downlink, RTT/RTT variation tests into the new MobiPerf.

Feature: App needs to display version information

version info needs to be on the splash page and on the "About Us" page.

Android support for anonymous data collection

We had this in the previous Mobiperf codebase (before reverting back to Speedometer) and it was addressed in this pull request:

#74

We should make the same change in the newest version of the code.

URLs for site creating stack traces

This is something that is unlikely to happen from average users, but IMO it poses two issues when it does happen:

You are exposing implementation (not a huge thing since MobiPerf is open source).
It looks sloppy and does not help the user do the right thing.

If I were to enter a URL such as http://openmobiledata.appspot.com/checkin

It should do something other than dump a stack trace to my browser. In most cases it should redirect to a help or FAQ instructing them about how to use the site.

In my case I was looking for JSON serialized data with the link '/timeseries/data'. Note that I am not saying that for development we shouldn't display the stack trace, but we should have a plan to do something more friendly for users.

Need to finalize anonymization pipeline

Feature: Battery threshold should be 75% by default

Current default is 90% which is too high to collect much data. Set to 75% by default.

directory structure

Maybe this is just me, but I'm using Eclipse to manage both the server code and the app code, and Eclipse doesn't like it when one project directory contains another project. What do you think about moving all the MobiPerf app code into an 'android' directory similar to the Speedometer repository?

UI feature: Need status page in app

The app needs a clear status page (or a panel on the main page) with basic information on the app's status:

How long it has been running
Whether background measurements are enabled
How many measurements have been taken
How many measurements have been uploaded (or alternately how many pending to be uploaded)
Last checkin time
Whether background tasks are disabled due to battery status
etc.

At present it is next to impossible to tell what state the app is in. The Speedometer app does all of this, and unless we get these things fixed soon I propose we revert back to Speedometer (renamed to "Mobiperf") for the beta release, since we seem to have had a lot of regression in functionality and stability since then.

Unit tests

Unit tests need to be updated so they can run.

Results temporarily disappear in the "Results" => "Manual" tab

To reproduce this bug, first run any test so that in the "Manual" tab, there is some results to show.

Then go to "Results" => "Manual", there should be some results.

Then rotate the screen so that the app adjusts itself to the new orientation.

Then all results in the "Manual" tab is gone. But if you click "Scheduled" and then click "Manual" tab again, the results will reappear.

Instead, we should show the results immediately after the screen is rotated, otherwise, the user might get confused. This is a minor bug of course. I didn't check whether this bug also exists in the "Scheduled" tab, but it's quite possible.

Put old MLab server code into /mlab

Put old MLab server code into /mlab
Start working on MLab server codes

mobiperf / mobiperf Goto Github PK

mobiperf's Introduction

mobiperf's People

Contributors

Stargazers

Watchers

Forkers

mobiperf's Issues

Recommend Projects

Recommend Topics

Recommend Org