Comments (12)
This is a really good question. It's an interesting and challenging problem to solve. You can register a framework listener and get alerted to exceptions that way. The trick would be synchronizing with the running test thread, and also knowing if the exception is related to your test. I think we could come up with something that would work though.
from osgi-test.
Normally when a callback thread calls code which throws an unexpected exception, the callback thread must decide how to handle the exception. It may log it. It may pass it to an uncaught exception handler. But an exception thrown in another thread will not be visible in the thread running the test case as there is no likely place for such an exception to be thrown from.
In the case of a framework callback thread, it will likely log it to the OSGi Log Service.
from osgi-test.
In the case of a framework callback thread, it will likely log it to the OSGi Log Service.
I suspect in most cases the framework will publish a FrameworkEvent error. That does end up getting published as a LogEvent if there is a log service implementation present.
from osgi-test.
While I get that a callbackthread should handle the exception itself, for testing it would be useful (if not needed) to catch runtime exceptions, after all, it is a testing framework to find problems.
Isn't there something possible with the bnd test launcher? Catch any exceptions in there and fail on them?
Alternatively, If those exception are published as FrameworkEvents, would it be possible to have an Extension that handles them and fails a test?
from osgi-test.
While I get that a callbackthread should handle the exception itself, for testing it would be useful (if not needed) to catch runtime exceptions, after all, it is a testing framework to find problems.
Isn't there something possible with the bnd test launcher? Catch any exceptions in there and fail on them?
Alternatively, If those exception are published as FrameworkEvents, would it be possible to have an Extension that handles them and fails a test?
Yes, that is possible. I was thinking something along those lines. One trick is that the framework events will not be delivered synchronously - the test that caused it may be finished by the time the corresponding framework event is received.
Also, if the framework hits a bump serious enough to cause an exception, there will likely be other side effects. You could test for those side effects.
I'm sure something is possible here - it will just require a bit of thought. I think @stbischof already has a draft pr for capturing and asserting on framework events - we may be able to (metaphorically) take that one out and dust it off.
from osgi-test.
If the test framework could do something to trigger a framework event post test then it could block on that event to happen (for example, call refreshPackages on a "dummy bundle" that would cause a PACKAGES_REFRESHED event). This would ensure any FrameworkEvent ERROR events got published and captured by the test framework. This way you could detect if any unexpected errors occurred during the test.
from osgi-test.
@abroekhuis could you provide a testable case that throws a framework error so we can start prototyping?
from osgi-test.
@abroekhuis could you provide a testable case that throws a framework error so we can start prototyping?
Sure, probably will take a few days, but I will make something!
from osgi-test.
Isn't there something possible with the bnd test launcher? Catch any exceptions in there and fail on them?
I don't think that is wise or possible. There can be many exceptions reported via framework listeners that should not result in the failure/termination of the launched runtime.
Alternatively, If those exception are published as FrameworkEvents, would it be possible to have an Extension that handles them and fails a test?
Yes, but (1) only you can know what exceptions should be considered test case failure vs other possible benign exceptions sent to framework listeners. There is also the timing in that events are delivered asynchronously to framework listeners. So any such extension would have to wait for some time in case an interesting exception may flow to framework listeners in order to correlate the exception with a specific test method. Such negative-test would add considerable time (1 sec?) to each test method. I suppose you could wait until the test class completes for wait for any interesting exceptions to flow, but then you would lose correlation to the test method which may have provoked the exception.
In general, this is a hard problem so solve in a callback based programming model. The callback caller must protect against unexpected exceptions from the callback as the callback caller has many more callbacks to call. And one ill-behaved callback must not break the system by callback caller's thread to die. OSGi Promises have this basic issue the handle it by delivering the exception to the thread's uncaught exception handler. Since the Promise user can control the threads used for callbacks, they can install an appropriate uncaught exception handler to detect these errors during testing.
from osgi-test.
Isn't there something possible with the bnd test launcher? Catch any exceptions in there and fail on them?
I don't think that is wise or possible. There can be many exceptions reported via framework listeners that should not result in the failure/termination of the launched runtime.
Alternatively, If those exception are published as FrameworkEvents, would it be possible to have an Extension that handles them and fails a test?
Yes, but (1) only you can know what exceptions should be considered test case failure vs other possible benign exceptions sent to framework listeners. There is also the timing in that events are delivered asynchronously to framework listeners. So any such extension would have to wait for some time in case an interesting exception may flow to framework listeners in order to correlate the exception with a specific test method. Such negative-test would add considerable time (1 sec?) to each test method. I suppose you could wait until the test class completes for wait for any interesting exceptions to flow, but then you would lose correlation to the test method which may have provoked the exception.
In general, this is a hard problem so solve in a callback based programming model. The callback caller must protect against unexpected exceptions from the callback as the callback caller has many more callbacks to call. And one ill-behaved callback must not break the system by callback caller's thread to die. OSGi Promises have this basic issue the handle it by delivering the exception to the thread's uncaught exception handler. Since the Promise user can control the threads used for callbacks, they can install an appropriate uncaught exception handler to detect these errors during testing.
These are the reasons why I was thinking it would be more appropriate to have a library (something like what @stbischof prototyped) to capture and assert on framework events, If you're interested in them then you can use this to look out for the exceptions that you might be interested in; if you're not then you can avoid the overhead of synchronizing with the framework thread.
from osgi-test.
This issue has been automatically marked as stale because it has not had recent activity. Given the limited bandwidth of the team, it will be automatically closed if no further activity occurs. If you feel this is something you could contribute, please have a look at our Contributor Guide. Thank you for your contribution.
from osgi-test.
This issue has been automatically closed due to inactivity. If you can reproduce this or if you have a good use case for this feature, please feel free to reopen the issue with steps to reproduce, a quick explanation of your use case or a high-quality pull request.
from osgi-test.
Related Issues (20)
- [assertj] Meta-assertions for bundles HOT 2
- Release 1.1.0 HOT 3
- [junit5] Simplify injector code
- [junit5] InstalledBundleExtension sanity checking
- Do not substitutably import the org.osgi.test.common.annotation package in junit5 HOT 9
- [ServiceExtension] Tweak wildcard type inferencing for upper bounds HOT 6
- [ServiceExtension] Type compatibility check for @InjectService.service() HOT 5
- [InjectingExtension] Refactoring suggestions HOT 10
- Add some descriptive documentation HOT 6
- osgi-test-example-mvn / org.osgi.test.example.player.test.PlayerTest are not executed HOT 4
- Need to obtain code coverage reports and interactive debugging within intellij and gradle plugin HOT 9
- Service instance are kept between different BundleContext's, Tests and test Classes HOT 4
- Provide a Junit5 Extension to start an OSGi-Connect-Framework as part of the test without any need for special setup HOT 4
- @ServiceSource should give a meaningfull error if no services are found HOT 3
- @ServiceSource should support ServiceAware<ServiceType> HOT 4
- readme link to codeql broken HOT 2
- NPE with @WithFactoryConfiguration HOT 15
- [cm] Support Systemproperties in Property HOT 4
- [cm] either create unbound config, or provide an option for it. HOT 3
- Weird behaviour when using `ConfigurationExtension` and standard config files HOT 10
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from osgi-test.