Comments (5)
oliverhu
commented on August 23, 2024
You are retrying the same testCase inside the same simulator. This looks more like a parsing error. @MattNewberry could you please paste your simulator log here? it is under ./output/ and with name N-simulator.log. cc @vargon
from bluepill.
ob
commented on August 23, 2024
Bluepill can run failed tests multiple times (to detect flaky tests) but that option (-failure-tolerance) is supposed to be off by default. It's not clear from the output that this is what's happening though... I thought we had a retry number in there. Can you try explicitly setting --error-retries to zero in the command line?
Here for your cutting & pasting convenience:
bluepill -a ./build/Unit\ Test-iphonesimulator/GuestCenter.app -s ./GuestCenter.xcodeproj/xcshareddata/xcschemes/Acceptance\ Tests1.xcscheme -o ./output/ -d "iPad Pro (9.7-inch)" -R 0
I just want to make sure we're not using the wrong timeout somewhere ;-)
from bluepill.
MattNewberry
commented on August 23, 2024
Without retries set to zero (original cmd)
(Of note, there are 2 failures within the test, could that be related the +3 count?)
Simulator log:
--------------------------------------------------------------------------------
Tests started: 2017-02-03 15:31:45 +0000
--------------------------------------------------------------------------------
2017-02-03 10:32:59.061 GuestCenter[21296:353235] KIFTester loaded
Test Suite 'Selected tests' started at 2017-02-03 10:33:00.106
Executed 0 tests, with 0 failures (0 unexpected) in 0.000 (0.000) seconds
Test Suite 'Acceptance Tests1.xctest' started at 2017-02-03 10:33:00.108
Test Suite 'MakeReservationStats' started at 2017-02-03 10:33:00.108
Test Case '-[Acceptance_Tests1.MakeReservationStats testGuest2WithNoStats]' started.
/Users/Matt/work/opentable/GuestCenter-iPad/Acceptance Tests/KIFTestActor+CommonSetup.m:126: error: -[Acceptance_Tests1.MakeReservationStats testGuest2WithNoStats] : The step timed out after 30.00 seconds: Could not find element with accessibilityIdentifier == "MenuButton"
/Users/Matt/work/opentable/GuestCenter-iPad/Acceptance Tests/MakeReservationStats.swift:18: error: -[Acceptance_Tests1.MakeReservationStats testGuest2WithNoStats] : Failure in child step: The step timed out after 30.00 seconds: Could not find element with accessibilityIdentifier == "MenuButton"
Test Case '-[Acceptance_Tests1.MakeReservationStats testGuest2WithNoStats]' failed (33.108 seconds).
Console output:
matthews-mbp:GuestCenter-iPad matt$ bluepill -a ./build/Intermediates/CodeCoverage/Products/Debug-iphonesimulator/GuestCenter.app -s ./GuestCenter.xcodeproj/xcshareddata/xcschemes/GuestCenter.xcscheme -o ./output/ -d "iPad Air 2" -S 5000 -T 5000 -J
[ INFO ] This is Bluepill v0.1.0-43-ga084f08
[ INFO ] Running with 4 simulators.
[ INFO ] Packed tests into 4 bundles
[ INFO ] Started Simulator 1 (PID 20899).
...
[ INFO ] Using 807 of 1064 processes.
[ FAILED ] (3) 32.599612s MakeReservationStats/testGuest2WithNoStats
[ FAILED ] (3) 32.601006s MakeReservationStats/testGuest2WithNoStats
[ FAILED ] (3) 33.089362s MakeReservationStats/testGuest2WithNoStats
With -R 0
No change... same behavior
from bluepill.
oliverhu
commented on August 23, 2024
It seems to an issue with regex parsing. @vargon is fixing that 🗡
from bluepill.
vargon
commented on August 23, 2024
Add PR #83 to handle this issue. @MattNewberry let me know if you can try out that branch and if it fixes your issue. Thanks!
from bluepill.
Related Issues (20)
- Xcode 13.1 Support
- [Reporting bug] Tests that passed on retry are not being reported accurately HOT 4
- Add GitHub CODEOWNERS
- Bluepill reporting issue HOT 2
- Shutting down simulator failed: Unable to shutdown device in current state: Shutdown
- Proposal to remove unsafe-skip-xcode-version-check HOT 2
- In bazel when having a test_host that ends with -App it will remove part of the name.
- Proposal: Allow Bluepill to run non-app hosted tests HOT 3
- Homebrew version of bluepill is out of date HOT 3
- Xcode 13.2.1 CRASH with "App crashed before tests started" HOT 1
- bluepill still getting the defualt path for xcode using xcode-path HOT 1
- Any plan for Xcode 13.4? HOT 2
- Need bluepill support for XCode13.4.1 HOT 8
- Timestamp in Junit xml in GMT format issue. HOT 1
- (BLUEPILL) Failed to read data for UIABCTests - ERROR: Failed to load some test bundles HOT 20
- `bundleIdentifier != nil` runtime crash HOT 3
- Xcode 14.0.1 NSInvalidArgumentException error
- Gather attachments from test run HOT 3
- Don't run disabled tests HOT 2
- Is there a plan to add support for XCUITest with real devices?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bluepill.