Use Better Screenshot Means about spoon HOT 32 OPEN

I just tried out this library, and it works like a charm with dialogs.

https://github.com/jraska/Falcon

from spoon.

We use the UiAutomation.takeScreenshot() method for our tests without problems. Not sure what you guys have different, but it never returned null or crashed the app for us. We use it in different real devices and we have something like 50+ test cases.

from spoon.

It just loops over all the windows like Robotium and smashes them together.
We can copy their implementation in or just link against the library at
worst.

On Sat, Mar 19, 2016, 8:56 AM Richard Radics [email protected]
wrote:

I just tried out this library, and it works like a charm with dialogs.

https://github.com/jraska/Falcon

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
#4 (comment)

from spoon.

Root detection added in jw/root branch.

from spoon.

Alternatively, try to use this: http://code.google.com/p/android-screenshot-library/ then we wouldn't need root!

from spoon.

Just thinking out loud: Would it be possible to take the screenshots via ddms? Somehow the instrumenation app would need to send a command up to the runner, so it could invoke IDevice.getScreenshot.

from spoon.

The test would also have to block until the screenshot was taken which is the harder problem.

from spoon.

I managed to get DDMS take screenshots. Tests are also blocking until the screenshot is taken.

I hooked up android-screenshot-lib (not the same as the one on google code). Basically it listens for logs with a special tag. To get the test to block, a file will be created on the device which gets deleted via adb. As soon as the file is gone the test continues. Root is obviously not needed on the device.

I didn't request a pull yet, because it's not fully tested yet (+I didn't write tests) and there is still one issue: screenshots appear to be duplicated in the resulting report. But I didn't find the cause yet. Also the screenshots are not rotated with respect to the current device orientation, but I actually don't mind that.

May I ask for a review? See my screenshot-ddms branch

from spoon.

@panzerfahrer We explicitly avoided going that route since it's very fragile and the blocking is awkward. You may have noticed that the library is where we stole the animated GIF creation from, however.

One way we were going to approach it was to start a web server on the desktop and use adb to port forward it to all of the devices. This way the Spoon.screenshot method could make a synchronous HTTP connection with the tag and test info and wait for the response.

The other would be to use a native service deployed via adb on test run which allows taking full screen captures on the device and then pulling at the end in the normal fashion.

For something like what you describe to be added we definitely would need things like proper orientation support. I'll try to take a look at what you have tonight.

from spoon.

I am following this issue as I would like to see dialogs in the screenshots. I am using Robotium in addition to Spoon, when I used their solo.takeScreenshot method it captured the dialog fine. However, it just captured the view for the dialog and not the entire screen. You probably don't like that approach as it would mess up the animated gif.

from spoon.

Correct. We also don't want to depend on Robotium!

We'll likely go the route of starting a native service over ADB and use it to take screenshots of the entire screen (including the status bar and soft buttons, woot).

from spoon.

Started work on the jw/full-screen-capture branch. The idea is that the runner deploys a binary that listens on a port for connections and a full path to the target image. The server captures the framebuffer and encodes it to a PNG at the destination file. If any exceptions are encountered they are sent back along the socket connection, otherwise it is simply closed.

AOSP has two on-device binaries for capturing the framebuffer screencap and screenshot. Both have dependencies on internal AOSP things which we probably want to tear out into a standalone tool. We also would need to get compilation working for all architectures and have detection (using #76) to deploy the correct binary. I'm fairly certain the maven-android-plugin supports this but we'd have to figure out how to bundle the results as part of the final runner jar.

We also need an effective way of not starting duplicate servers and killing the server once execution has completed. A simple PID file should suffice.

I don't have the time to work on this now. Maybe in 4-6 weeks.

from spoon.

After my first attempt with the quirky "lock file" approach, I tried it with running a server on the device.

I'm starting a server within the InstrumentationTestRunner. spoon-runner houses a client which connects to the server via an ADB port forwarding. I'm not using a native service, but plain Java socket channels.

Unfortunately, it's not really working and I couldn't yet figure out what I'm missing. Anyhow, maybe it can be of help. I had to postpone my work on this, so I'm not sure when can get a look into it again.

panzerfahrer/screenshot-ddms-socket

from spoon.

@panzerfahrer That's the route we initially wanted to go as well. Haven't really had time to explore it. If I get back around to work on it before you do I will definitely take a look. Thanks!

from spoon.

BOOM https://developer.android.com/reference/android/app/UiAutomation.html#takeScreenshot()

from spoon.

New in 4.3 eh?

from spoon.

Has there been any progress on the issue of not being able to see dialogs in screenshots?

from spoon.

No. Free time is fleeting nowadays.

from spoon.

But we do accept pull requests.

from spoon.

Thanks for the reply! No problem I understand. We're looking to get it working for a project at work so I'll see if I can get the time to fix this.

Looking at the code inside the takeScreenshot method of Robotium it seems at first glance like a more sophisticated version of what you use in Spoon. Would it be ok if we tried to port their stuff over into Spoon? There wouldn't be any dependency, I'd just move their code over to spoon.

from spoon.

You mean the gross, grab-all-the-windows-with-reflection-and-render-them-in-reverse method? I suppose there's worse things... go for it.

from spoon.

Pretty much. The goal for us to to have screenshots which show things like dialogs as well as the foreground activity. Ideally we'd just use the UIAutomation class but we'd like it to work on older devices too.

I'll take a look and see what I can do with it.

from spoon.

Here is some code that I wrote. I have a higher level automation api which encapsulates both Spoon and Robotium. I should probably do a pull request.

Two methods for taking screenshots:

void takeScreenshot(String tag);
void takeScreenshot(String tag, boolean isDialog);

First one will call Spoon's method. Second flavor calls the ScreenshotTaker class below. You can see that I put the screenshots in the same folder as Spoon, and I use the same convention for naming the files. This way when the test is run, Spoon pulls the screenshots taken by both methods.

        final String imageName = System.currentTimeMillis() + "_" + tag;
        final String imageFolder = String.format("%s%s/%s/",
                ScreenshotTaker.SPOON_SCREEN_SHOTS_LOCATION,
                mClassName,
                mMethodName);

        runOnMainSync(new Runnable() {
            @Override
            public void run() {
                try {
                    ScreenshotTaker screenshotTaker = new ScreenshotTaker(imageFolder);
                    screenshotTaker.takeScreenShot(imageName);
                } catch (Exception e) {
                    logMsg("Error taking screenshot: ", e.getMessage());
                }
            }});

ScreenshotTaker class which has pretty much the same code as what Robotium uses...

import android.graphics.Bitmap;
import android.view.View;

import java.io.File;
import java.io.FileOutputStream;
import java.lang.reflect.Field;

import static com.evernote.test.framework.TestLogger.logTestError;

public class ScreenshotTaker {

private String screenShotFolderLocation;
private String windowManagerString;

public static final String SPOON_SCREEN_SHOTS_LOCATION = "/data/data/com.mycompany/app_spoon-screenshots/";


public ScreenshotTaker(String folderLocation){
    screenShotFolderLocation = folderLocation;
    setWindowManagerString();
}

public void takeScreenShot(String tag) throws Exception {
    takeScreenShot(getRecentDecorView(getWindowDecorViews()), tag);
}

public void takeScreenShot(View view, String name) throws Exception {

    if (view == null) {
        logTestError("takeScreenShot: view from getWindowDecorViews() is null.");
        return;
    }

    File sddir = new File(screenShotFolderLocation);
    if (!sddir.exists()) {
        logTestError("Spoon screenshot folder does not exist: %s", screenShotFolderLocation);
        return;
    }

    view.setDrawingCacheEnabled(true);
    view.buildDrawingCache();
    Bitmap b = view.getDrawingCache();
    File screenShotFile = new File(screenShotFolderLocation + name + ".png");

    FileOutputStream fos = new FileOutputStream(screenShotFile);
    if (fos != null) {
        b.compress(Bitmap.CompressFormat.PNG, 100, fos);
        screenShotFile.setReadable(true, false);

        fos.flush();
        fos.close();
    }
}


/**
 * Returns the WindorDecorViews shown on the screen.
 *
 * @return the WindorDecorViews shown on the screen
 */

public View[] getWindowDecorViews()
{

    Field viewsField;
    Field instanceField;
    try {
        viewsField = windowManager.getDeclaredField("mViews");
        instanceField = windowManager.getDeclaredField(windowManagerString);
        viewsField.setAccessible(true);
        instanceField.setAccessible(true);
        Object instance = instanceField.get(null);
        return (View[]) viewsField.get(instance);
    } catch (SecurityException e) {
        e.printStackTrace();
    } catch (NoSuchFieldException e) {
        e.printStackTrace();
    } catch (IllegalArgumentException e) {
        e.printStackTrace();
    } catch (IllegalAccessException e) {
        e.printStackTrace();
    }
    return null;
}

/**
 * Returns the most recent DecorView
 *
 * @param views the views to check
 * @return the most recent DecorView
 */

public final View getRecentDecorView(View[] views) {
    if (views == null) {
        logTestError("Error in getRecentDecorView: 0 views passed in.");
        return null;
    }

    final View[] decorViews = new View[views.length];
    int i = 0;
    View view;

    for (int j = 0; j < views.length; j++) {
        view = views[j];
        if (view != null && view.getClass().getName()
                .equals("com.android.internal.policy.impl.PhoneWindow$DecorView")) {
            decorViews[i] = view;
            i++;
        }
    }
    return getRecentContainer(decorViews);
}

/**
 * Returns the most recent view container
 *
 * @param views the views to check
 * @return the most recent view container
 */

private final View getRecentContainer(View[] views) {
    View container = null;
    long drawingTime = 0;
    View view;

    for(int i = 0; i < views.length; i++){
        view = views[i];
        if (view != null && view.isShown() && view.hasWindowFocus() && view.getDrawingTime() > drawingTime) {
            container = view;
            drawingTime = view.getDrawingTime();
        }
    }
    return container;
}


private static Class<?> windowManager;
static{
    try {
        String windowManagerClassName;
        if (android.os.Build.VERSION.SDK_INT >= 17) {
            windowManagerClassName = "android.view.WindowManagerGlobal";
        } else {
            windowManagerClassName = "android.view.WindowManagerImpl";
        }
        windowManager = Class.forName(windowManagerClassName);

    } catch (ClassNotFoundException e) {
        throw new RuntimeException(e);
    } catch (SecurityException e) {
        e.printStackTrace();
    }
}


/**
 * Sets the window manager string.
 */
private void setWindowManagerString(){

    if (android.os.Build.VERSION.SDK_INT >= 17) {
        windowManagerString = "sDefaultWindowManager";

    } else if(android.os.Build.VERSION.SDK_INT >= 13) {
        windowManagerString = "sWindowManager";

    } else {
        windowManagerString = "mWindowManager";
    }
}

}

from spoon.

Any plans for "UiAutomation.html#takeScreenshot()" support ( with fallback for older API versions )?

Also, it would be great if there was a Spoon.logImage(String tag, Bitmap bitmap) which puts any bitmap in the Spoon report. This could be used for testing a standalone custom view and for snapshot Animations in progress.

from spoon.

Just tried quickly hacking in takeScreenshot support. Took about 5 minutes, but it turns out takeScreenshot occasionally (and for no reason I can predict..) returns null.

(We had a fun issue where our tests would fail due to a "please update your google play services version" dialog. All our screenshots looked perfect, but when we ran without headless mode we could spot the huge dialog over the entire screen :P )

from spoon.

#175

Better than the decorview on devices that supports it atleast.

from spoon.

Caution, UiAutomation.html#takeScreenshot() is flaky and can sometimes cause issues with OpenGL memory allocations... We have had cases where it causes other apps to crash ( like the launcher ), presumably because it is triggering system wide issues with OpenGL memory allocations.

from spoon.

UiAutomation.takeScreenshot() is so broken, for me every time it either returns null on the emulator or crashes the process on a real device...
I guess the only viable option is a screenshot service (like this one)!

from spoon.

There's other means of taking a screenshot like MediaProjection. We tried those services with Spoon a year or two ago and they were pretty awful.

from spoon.

@RichardGuion Thank you for your help ! It's working for me 👍

from spoon.

I am using UiAutomation.takeScreenshot() without any issues.. with spoon 1.7.1 don't like the fact that I have to click on a link in the report....
#501 allows spoon to show any user defined png in the report... I think it allows for any implementation you like!

from spoon.

I use both Spoon library and Falcon library and it works well.
In Gradle:
androidTestImplementation 'com.squareup.spoon:spoon-client:1.7.1'
//Spoon is really great library, but it also does not include dialogs and other extra windows in its screenshots
//Falcon support capturing dialog: https://github.com/jraska/Falcon
androidTestImplementation 'com.jraska:falcon:2.1.1'
androidTestImplementation 'com.jraska:falcon-spoon-compat:2.1.1'

And use this code to capture dialog:
Falcon.takeScreenshot(activity, file)

from spoon.