Giter Club home page Giter Club logo

flauinspect's Introduction

Alt text

Badges

What Badge
Chat Join the chat at https://gitter.im/FlaUI/Lobby
Build Build status
Tests AppVeyor tests
Libraries (NuGet) Nuget Nuget Nuget
CI Artefacts FlaUI CI

Introduction

FlaUI is a .NET library which helps with automated UI testing of Windows applications (Win32, WinForms, WPF, Store Apps, ...).
It is based on native UI Automation libraries from Microsoft and therefore kind of a wrapper around them.
FlaUI wraps almost everything from the UI Automation libraries but also provides the native objects in case someone has a special need which is not covered (yet) by FlaUI.
Some ideas are copied from the UIAComWrapper project or TestStack.White but rewritten from scratch to have a clean codebase.

Sponsoring

If you appreciate my work, feel free to support me by sponsoring on github or with a one-time payment over at PayPal.

Why another library?

There are quite some automation solutions out there. Commercial ones like TestComplete, Ranorex, CodedUI just to name a few. And also free ones which are mainly TestStack.White.
All of them are based on what Microsoft provides. These are the UI Automation libraries. There are three versions of it:

  • MSAA
    • MSAA is very obsolete and we'll skip this here (some like CodedUI still use it)
  • UIA2: Managed Library for native UI Automation API
    • UIA2 is managed only, which would be good for C# but it does not support newer features (like touch) and it also does not work well with WPF or even worse with Windows Store Apps.
  • UIA3: Com Library for native UI Automation API
    • UIA3 is the newest of them all and works great for WPF / Windows Store Apps but unfortunately, it can have some bugs with WinForms applications (see FAQ) which are not existent in UIA2.

So, the commercial solutions are mostly based on multiple of those and/or implement a lot of workaround code to fix those issues. TestStack.White has two versions, one for UIA2 and one for UIA3 but because of the old codebase, it's fairly hard to bring UIA3 to work. For this, it also uses an additional library, the UIAComWrapper which uses the same naming as the managed UIA2 and wraps the UIA3 com interop with them (one more source for errors). FlaUI now tries to provide an interface for UIA2 and UIA3 where the developer can choose, which version he wants to use. It should also provide a very clean and modern codebase so that collaboration and further development is as easy as possible.

Usage

Installation

To use FlaUI, you need to reference the appropriate assemblies. So you should decide, if you want to use UIA2 or UIA3 and install the appropriate library from NuGet. You can of course always download the source and compile it yourself.

Usage in Code

The entry point is usually an application or the desktop so you get an automation element (like a the main window of the application). On this, you can then search sub-elements and interact with them. There is a helper class to launch, attach or close applications. Since the application is not related to any UIA library, you need to create the automation you want and use it to get your first element, which then is your entry point.

using FlaUI.UIA3;

var app = FlaUI.Core.Application.Launch("notepad.exe");
using (var automation = new UIA3Automation())
{
	var window = app.GetMainWindow(automation);
	Console.WriteLine(window.Title);
	...
}
using FlaUI.Core.AutomationElements;
using FlaUI.UIA3;

// Note: Works only pre-Windows 8 with the legacy calculator
var app = FlaUI.Core.Application.Launch("calc.exe");
using (var automation = new UIA3Automation())
{
	var window = app.GetMainWindow(automation);
	var button1 = window.FindFirstDescendant(cf => cf.ByText("1"))?.AsButton();
	button1?.Invoke();
	...
}

Further Resources

YouTube Tutorials

Have a look at H Y R Tutorials. This channel provides some videos to get you started with FlaUI.

FlaUI UITests

FlaUI itself contains quite some UI tests itself. Browse to the code of them here and look how they work.

Chat

Head over to the chat to ask your specific questions.

Contribution

Feel free to fork FlaUI and send pull requests of your modifications.
You can also create issues if you find problems or have ideas on how to further improve FlaUI.

Donors and Sponsors

  • Thank you Gehtsoft USA LLC for the generous donation

Acknowledgements

JetBrains

Thanks to JetBrains for providing a free license of ReSharper.

AppVeyor

Thanks to AppVeyor for providing a free CI build system for FlaUI.

TestStack.White

Thanks to the creators and maintainers (especially to @JakeGinnivan and @petmongrels) for their work and inspiration for this project.

Microsoft

Thanks to Microsoft for providing great tools which made developing this project possible.

flauinspect's People

Contributors

aurax avatar martinmoenks avatar roemer avatar sean-sel avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

flauinspect's Issues

Application Unresponsive

The Application often becomes very unresponsive in Focus Tracking mode, or even just expanding the Desktop initial Pane, taking seconds or even minutes to do anything. This happens in both modes UIA2 and UIA3.

I finally gave up and closed it.

Instructions Not Clear

Please Update the readme.md to include information that you can get this lovely tool through Chocolatey, I spent literal hours trying to download this and compile on my own only to realise I somehow skipped .NET 4.7.1 and couldn't install it and tried to compile it myself. I Even looked at Cake and so on. It'd be a literal godsend to future users if you could add this little bit of information.

Install Chocolatey here using your Windows Powershell:
https://chocolatey.org/install

Then in cmd or Power shell do:
"choco install flauinspect"

can be run in cmd by just typing:
"flauinspect"

or by going to the chocolatey install folder and locating the .exe there.

AutomationId property displays the name of the item if there is no AutomationId set

I am seeing some odd behavior with FlaUInspect. I have an element I am trying to find, it has a name of "listView" but there is no AutomationId set for it. When I look at this item in FlaUInspect, it says the AutomationId is "listView" and the name property is blank. If I set an AutomationId for the element, the correct AutomationId shows up, but name looks like it is still blank. The element is using x:Name to set the name.

I am not sure if this actually impacts grabbing the element with FlaUI. I specified an AutomationId of "listView" and I got back an element, but it was a different element and I didn't dig into it to see exactly which element was being returned and if that element did in fact have "listView" as it's AutomationId.

Can't build FlaUInspect

Hi,

Sorry if I'm missing something obvious, but I can't build FlaUInspect.

The errors are:

Severity Code Description Project File Line Suppression State
Error CS1003 Syntax error, ',' expected FlaUInspect E:\projects\Misc\FlaUInspect\src\FlaUInspect\ViewModels\ElementViewModel.cs 77 Active
Error CS0206 A property or indexer may not be passed as an out or ref parameter FlaUInspect E:\projects\Misc\FlaUInspect\src\FlaUInspect\ViewModels\ElementViewModel.cs 77 Active
Error CS0103 The name 'value' does not exist in the current context FlaUInspect E:\projects\Misc\FlaUInspect\src\FlaUInspect\ViewModels\ElementViewModel.cs 77 Active
Error CS0103 The name 'value' does not exist in the current context FlaUInspect E:\projects\Misc\FlaUInspect\src\FlaUInspect\ViewModels\ElementViewModel.cs 77 Active

The offending line of code is:

public ControlType ControlType => AutomationElement.Properties.ControlType.TryGetValue(out ControlType value) ? value : ControlType.Custom;

Specifically, it's about out ControlType value.

I haven't been programming in C# for some time, so I'm not up-to-date with the syntax, and am only superficially familiar with the =>-notation. So again, sorry if I'm missing something obvious.

Can you help?

Thanks, regards, Miel.

Prefer32Bit should be false

The <Prefer32Bit> flag in FlaUIInspect.csproj is left at default (true) in release mode (and curiously set to true in Debug mode).

If the "standard scenario" is to inspect x64 applications when on 64bit Windows, the flag would be better set to false so FlaUIInspect starts as x64.

Since there are some known issues inspecting x64 apps with FlaUIInspect running as x86, and the Choclatey build is using AnyCPU+Prefer32Bit so it's running x86, users will often have to compile themselves to avoid that issue.

Happy to submit a PR for this if this sounds like a good change.

FlaUInspect (UIA2) Wrong detection of TabItems with .net6

After the migration of a winforms application from .net4.8 to .net6 we have many FlaUI tests failing. After analysis, we found out that the TabItems are not identified correctly in .net6. We could also reproduce this behavior with .net8
image

We were able to reproduce the issue with FlaUInspect with a minimal application (attached).
WinForms.zip

FlaUInspect (UIA3) doesn't see some windows

It seems FlaUInspect (in UIA3 mode) may ignore some top-level windows.
Here is the case with RogueKiller Anti-Malware (Free) 15.12.0.0 64 bits when it runs in Windows 11:
image

Window class name: Qt5152QWindowIcon
Probably it is an issue in the underlying library FlaUI.

Build error

When i run code with visual, it show some error:

Error CS0104 'AutomationElement' is an ambiguous reference between 'System.Windows.Automation.AutomationElement' and 'FlaUI.Core.AutomationElements.AutomationElement' FlaUInspect D:\XXX\FlaUInspect-master\FlaUInspect-master\src\FlaUInspect\ViewModels\ElementViewModel.cs 23

Error CS0104 'ControlType' is an ambiguous reference between 'System.Windows.Automation.ControlType' and 'FlaUI.Core.Definitions.ControlType' FlaUInspect D:\XXX\FlaUInspect-master\FlaUInspect-master\src\FlaUInspect\ViewModels\ElementViewModel.cs 81

I don't know exactly where reference

Allow Raw-Walking with FlaUInspect

Currently, FlaUInspect supports only viewing control elements. Rewrite it so that it uses the TreeWalker and allow switching the TreeWalker to Raw or Content.

Handle some runtime exceptions which appear during interaction with Telerik WPF based apps

Two exceptions were observed:

  1. \src\FlaUInspect\Core\HoverMode.cs, line 43:
    var hoveredElement = _automation.FromPoint(screenPos);
System.IO.FileNotFoundException
  HResult=0x80070002
  Message=The system cannot find the file specified. (Exception from HRESULT: 0x80070002)
  1. \src\FlaUInspect\ViewModels\ElementViewModel.cs, line 298:
    var foreColor = (int)pattern.DocumentRange.GetAttributeValue(TextAttributes.ForegroundColor);
System.InvalidCastException
  HResult=0x80004002
  Message=Specified cast is not valid.
  Source=FlaUInspect
  StackTrace:
   at FlaUInspect.ViewModels.ElementViewModel.LoadDetails() in <FlaUInspect>\src\FlaUInspect\ViewModels\ElementViewModel.cs:line 298

UI elements no longer detectable after navigation

Hi.

We have experienced, that clearly detectable UI elements suddenly are no longer detectable after performing some navigation forth and back. The team have not been able to explain why, but have created a small app that easily reproduces the problem.

To reproduce, start the attached solution. Verify you can track the "Test FlaUI" in the data table.
Click the top "TestFlaUI" button in the data grid then click "Back". Do this twice.
Refresh the Inspector.
Track the "TestFlaUI" button.

Nothing is visible under the Tab page and no amount of refreshing the Inspector will make them visible.
If you only do the navigation once and refresh, then the button is trackable, it only breaks on the 2nd navigation. You can keep on tracking the UI element as long as you only do the navigation once between Inspector refreshes...

Hope you can help me figure out why the Inspector breaks.

Best regards
Thor
FlaUI_DataGridButtons.zip

Support for TechnologyName property

Is it possible to show the TechnologyName property for the elements?
See a screenshot by DevExpress

For all common controls/elements there should be "MSAA" or "Web" or "UIA".

This is required in order to validate whether third party extensions for UIA (such as from DevExpress) are loaded.

Inspector crash

While working with FlaUInspector we experience a lot of crashes when Hover Mode is enabled. Mainly related to writing code while as CTRL is used a lot.

One of the errors I got looked like this
An event was unable to invoke any of the subscribers (Exception from HRESULT: 0x80040201)
at Interop.UIAutomationClient.CUIAutomation8Class.CompareElements(IUIAutomationElement el1, IUIAutomationElement el2)
at FlaUI.UIA3.UIA3Automation.Compare(AutomationElement element1, AutomationElement element2)
at FlaUI.Core.AutomationElements.AutomationElement.Equals(AutomationElement other)
at FlaUInspect.ViewModels.MainViewModel.<>c__DisplayClass44_0.b__0(ElementViewModel child) in C:\GIT\Repositories\FlaUInspect\src\FlaUInspect\ViewModels\MainViewModel.cs:line 198
at System.Linq.Enumerable.FirstOrDefault[TSource](IEnumerable1 source, Func2 predicate)
at FlaUInspect.ViewModels.MainViewModel.FindElement(ElementViewModel parent, AutomationElement element) in C:\GIT\Repositories\FlaUInspect\src\FlaUInspect\ViewModels\MainViewModel.cs:line 198
at FlaUInspect.ViewModels.MainViewModel.ElementToSelectChanged(AutomationElement obj) in C:\GIT\Repositories\FlaUInspect\src\FlaUInspect\ViewModels\MainViewModel.cs:line 172
at FlaUInspect.Core.HoverMode.DispatcherTimerTick(Object sender, EventArgs e) in C:\GIT\Repositories\FlaUInspect\src\FlaUInspect\Core\HoverMode.cs:line 56
at System.Windows.Threading.DispatcherTimer.FireTick(Object unused)
at System.Windows.Threading.ExceptionWrapper.InternalRealCall(Delegate callback, Object args, Int32 numArgs)
at System.Windows.Threading.ExceptionWrapper.TryCatchWhen(Object source, Delegate callback, Object args, Int32 numArgs, Delegate catchHandler)
at System.Windows.Threading.DispatcherOperation.InvokeImpl()
at System.Windows.Threading.DispatcherOperation.InvokeInSecurityContext(Object state)
at MS.Internal.CulturePreservingExecutionContext.CallbackWrapper(Object obj)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at MS.Internal.CulturePreservingExecutionContext.Run(CulturePreservingExecutionContext executionContext, ContextCallback callback, Object state)
at System.Windows.Threading.DispatcherOperation.Invoke()
at System.Windows.Threading.Dispatcher.ProcessQueue()
at System.Windows.Threading.Dispatcher.WndProcHook(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
at MS.Win32.HwndWrapper.WndProc(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
at MS.Win32.HwndSubclass.DispatcherCallbackOperation(Object o)
at System.Windows.Threading.ExceptionWrapper.InternalRealCall(Delegate callback, Object args, Int32 numArgs)
at System.Windows.Threading.ExceptionWrapper.TryCatchWhen(Object source, Delegate callback, Object args, Int32 numArgs, Delegate catchHandler)
at System.Windows.Threading.Dispatcher.LegacyInvokeImpl(DispatcherPriority priority, TimeSpan timeout, Delegate method, Object args, Int32 numArgs)
at MS.Win32.HwndSubclass.SubclassWndProc(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam)
at MS.Win32.UnsafeNativeMethods.DispatchMessage(MSG& msg)
at System.Windows.Threading.Dispatcher.PushFrameImpl(DispatcherFrame frame)
at System.Windows.Threading.Dispatcher.PushFrame(DispatcherFrame frame)
at System.Windows.Application.RunDispatcher(Object ignore)
at System.Windows.Application.RunInternal(Window window)
at System.Windows.Application.Run(Window window)
at System.Windows.Application.Run()
at FlaUInspect.App.Main()

We also get a crash, if we select an item in the inspector, that is no longer present in the UI, like from a closed dropdown curtain.

My quesiton is, do anyone have an good suggestions or ideas on how to best solve these crashes? Other than remembering to turn off tracking after each use :).
Some crashes are due to a cached element not being there anymore and I was thinking about catching that and send back an empty AutomationElement. Then the UI would be void of property values, indicating to the user (s)he needs to refresh.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.