Giter Club home page Giter Club logo

analytics-apim's People

Contributors

chamilaadhi avatar chamindias avatar chamndesilva avatar dilini-muthumala avatar irham0019 avatar ketharan avatar ksdperera avatar maheshakya avatar malinthaprasan avatar mushir33 avatar nirmal070125 avatar niveathika avatar pamodaaw avatar praminda avatar rajeev3001 avatar raveensr avatar ruks avatar ruwiniwj avatar sacjaya avatar sajithshn avatar sandalu95 avatar shilmyhasan avatar swsachith avatar tanyam avatar tharindu1st avatar thidasala avatar upul avatar vithulan avatar wasuradananjith avatar wso2-jenkins-bot avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

analytics-apim's Issues

[Alert] Introduce suppression to avoid alert flooding

Description:

It was seen that the suppression has not been introduced to Abnormal response and backend time alerts and abnormal request count alerts. Hence the users will get alerts flooding when the alerts are enabled.
If user is getting the same alert again and again, it should not be sent for a specific time period.
Suggested Labels:

Suggested Assignees:

Affected Product Version:

OS, DB, other environment details and versions:

Steps to reproduce:
When you generate alert for each of the aforementioned alert time, the same alert will be generated again and again.

Related Issues:

Abnormal API Usage Alert template config name is confusing

Description:

In alert table there is an alert named abnormal api usage. The template config for this particular alert is AbnormalTierUsageAlert. We should be changing the naming to avoid confusion.

Suggested Labels:

Suggested Assignees:

Affected Product Version:
2.1.0
OS, DB, other environment details and versions:

Steps to reproduce:

Related Issues:

[AMHACK] [Manage Alert Types] Email Validation is not triggered

Description:
In API publisher, Manage Alert Types while configuring if a user provides a random string rather an email address it is still allowed to be saved. Can we have an email validation on save?

Suggested Labels:
Improvement

Affected Product Version:
APIM wum updated - wso2am-2.1.0.1510928473100.zip

OS, DB, other environment details and versions:
Standalone
Ubuntu 14
JDK - jdk1.8.0
DB - h2
Browser - Chrome Version 62.0.3202.62

Steps to reproduce:
Login to api publisher
Navigate to - Manage Alert Types and select an alert and provide an invalid email

Expected result
A validation message should be shown for invalid email id

API usage statistics fluctuating

While doing a PoC for a customer I've noticed that the API usage statistics are fluctuating. When I check the usage data in the API_REQUEST_SUMMARY table the total request count for an API changes, having an impact on the statistics shown in the Store and the Publisher.

This issue happens on different types of set-up. I've noticed this when API Manager 2.0.0 and the Analytics were installed on the same machine using H2 database. It also happened when I switched to a MySql database.
I now have an installation where APIM 2.0.0 and the Analytics run on a different server, using MySql as database.

In the example below you can see that today (August 31) the total amount of requests on August 29 changes.

When I executed the query the first time the total amount of requests for the Monuments 1.0.0 API by user monuments_client is 10 (6 + 4).

mysql> select * from API_REQUEST_SUMMARY;
+-----------+---------------------------+---------+--------------+------------------------------+-------------------------------+------------------+------------------+---------------------+---------------+------+-------+-----+------------------+
| api       | api_version               | version | apiPublisher | consumerKey                  | userId                        | context          | max_request_time | total_request_count | hostName      | year | month | day | time             |
+-----------+---------------------------+---------+--------------+------------------------------+-------------------------------+------------------+------------------+---------------------+---------------+------+-------+-----+------------------+
| Monuments | creator--Monuments:v1.0.0 | 1.0.0   | creator      | IYkOkwfISxpVHvH596DWfxLTErca | [email protected] | /monuments/1.0.0 |    1472479494000 |                   6 | 192.168.36.72 | 2016 |     8 |  29 | 2016-08-29 16:04 |
| Monuments | creator--Monuments:v1.0.0 | 1.0.0   | creator      | IYkOkwfISxpVHvH596DWfxLTErca | [email protected] | /monuments/1.0.0 |    1472626117000 |                   1 | 192.168.36.72 | 2016 |     8 |  31 | 2016-08-31 08:48 |
| Monuments | creator--Monuments:v1.0.0 | 1.0.0   | creator      | IYkOkwfISxpVHvH596DWfxLTErca | [email protected] | /monuments/1.0.0 |    1472477680000 |                   4 | wso2-openapi2 | 2016 |     8 |  29 | 2016-08-29 15:34 |
+-----------+---------------------------+---------+--------------+------------------------------+-------------------------------+------------------+------------------+---------------------+---------------+------+-------+-----+------------------+
3 rows in set (0.00 sec)

About ten minutes later (after doing some new requests, on August 31) the total of requests is 7 (6 + 1).

mysql> select * from API_REQUEST_SUMMARY;
+-----------+---------------------------+---------+--------------+------------------------------+-------------------------------+------------------+------------------+---------------------+---------------+------+-------+-----+------------------+
| api       | api_version               | version | apiPublisher | consumerKey                  | userId                        | context          | max_request_time | total_request_count | hostName      | year | month | day | time             |
+-----------+---------------------------+---------+--------------+------------------------------+-------------------------------+------------------+------------------+---------------------+---------------+------+-------+-----+------------------+
| Monuments | creator--Monuments:v1.0.0 | 1.0.0   | creator      | IYkOkwfISxpVHvH596DWfxLTErca | [email protected] | /monuments/1.0.0 |    1472479494000 |                   6 | 192.168.36.72 | 2016 |     8 |  29 | 2016-08-29 16:04 |
| Monuments | creator--Monuments:v1.0.0 | 1.0.0   | creator      | IYkOkwfISxpVHvH596DWfxLTErca | [email protected] | /monuments/1.0.0 |    1472626272000 |                   2 | 192.168.36.72 | 2016 |     8 |  31 | 2016-08-31 08:51 |
| Monuments | creator--Monuments:v1.0.0 | 1.0.0   | creator      | IYkOkwfISxpVHvH596DWfxLTErca | [email protected] | /monuments/1.0.0 |    1472475259000 |                   1 | wso2-openapi2 | 2016 |     8 |  29 | 2016-08-29 14:54 |
+-----------+---------------------------+---------+--------------+------------------------------+-------------------------------+------------------+------------------+---------------------+---------------+------+-------+-----+------------------+
3 rows in set (0.00 sec)

Again some 20 minutes later, the amount is 8 (4+4).

mysql> select * from API_REQUEST_SUMMARY;
+-----------+---------------------------+---------+--------------+------------------------------+-------------------------------+------------------+------------------+---------------------+---------------+------+-------+-----+------------------+
| api       | api_version               | version | apiPublisher | consumerKey                  | userId                        | context          | max_request_time | total_request_count | hostName      | year | month | day | time             |
+-----------+---------------------------+---------+--------------+------------------------------+-------------------------------+------------------+------------------+---------------------+---------------+------+-------+-----+------------------+
| Monuments | creator--Monuments:v1.0.0 | 1.0.0   | creator      | IYkOkwfISxpVHvH596DWfxLTErca | [email protected] | /monuments/1.0.0 |    1472479494000 |                   4 | 192.168.36.72 | 2016 |     8 |  29 | 2016-08-29 16:04 |
| Monuments | creator--Monuments:v1.0.0 | 1.0.0   | creator      | IYkOkwfISxpVHvH596DWfxLTErca | [email protected] | /monuments/1.0.0 |    1472626117000 |                   1 | 192.168.36.72 | 2016 |     8 |  31 | 2016-08-31 08:48 |
| Monuments | creator--Monuments:v1.0.0 | 1.0.0   | creator      | IYkOkwfISxpVHvH596DWfxLTErca | [email protected] | /monuments/1.0.0 |    1472477680000 |                   4 | wso2-openapi2 | 2016 |     8 |  29 | 2016-08-29 15:34 |
+-----------+---------------------------+---------+--------------+------------------------------+-------------------------------+------------------+------------------+---------------------+---------------+------+-------+-----+------------------+
3 rows in set (0.00 sec)

[APIM-Analytics][2.1.0] - API Usage By Resource Path Stat Mismatch

Description:

When you invoke resource of an API(eg: order/{orderId} ), with different path parameter, the stat shown in APIM does not give correct values.
Suggested Labels:

2.2.0, apim-analytics

Suggested Assignees:

Affected Product Version:
2.1.0
OS, DB, other environment details and versions:

Steps to reproduce:

  1. Configure APIM with API- analytics pack. Start up both servers.
  2. Publish sample API.
  3. Go to store.
  4. Try GET order/{orderId} with different IDs.
  5. Go to publisher analytics and check API usage by resource path.
  6. You will see the shown amount is less than the actual value.

Related Issues:

GeoLocation script execution results in errors with Oracle

Description:

geo-location script is resulting in errors when it is trying to insert empty string to Oracle DB. The reason is that oracle treats empty string as NULL. Since the columns which it tries to insert null is a primary key, it results in errors.

Suggested Labels:

Suggested Assignees:

Affected Product Version:
2.1.0, 2.2.0
OS, DB, other environment details and versions:
Oracle Database
Steps to reproduce:

Related Issues:

[Alerts] IP white-listing feature for unusual IP access alerts

Description:

When you configure alerts for unusual IP access alerts, it sends alerts for all IPs if the conditions are met. There could be a scenario that they would not like to get these alerts for specific APIs. To mitigate this there could a field that we can configure which IPs to white-list such that it would not send an alert for these IPs.
Suggested Labels:

alerts, ip-whitelisting, unusualIPaccessAlerts
Suggested Assignees:

Affected Product Version:
2.0.0, 2.1.0
OS, DB, other environment details and versions:

Steps to reproduce:

Related Issues:

Abnormal Request Count Alert does not send emails

Description:
Abnormal Request Count Alert does not send emails but it is visible in the Admin alert dashboard.

Suggested Labels:

Suggested Assignees:

Affected Product Version:

OS, DB, other environment details and versions:

Steps to reproduce:

Related Issues:

UX Scenario to consider

Description:
When a user accesses the analytics some may want to by default view "My APIs" first rather than "All APIs" but some may want the current view. Can we have a way for the users to configure this for their products so that they dont want to allways click on "MyAPIs" for each analytics graph.

This way we can improve the User experience and cater based on the customer needs.

Throttle and Fault stream analytics for microgateway is not supported

Description:

The microgateway analytics receiver has only been applied to the core stream. It has to be applied to fault and throttle streams as well.
Suggested Labels:

Suggested Assignees:

Affected Product Version:
2.6.0-alpha
OS, DB, other environment details and versions:

Steps to reproduce:

Related Issues:

Abnormal Request Count Email Alert doesn't work

Description:
Even though the alert is generated the email doesn't trigger.

Suggested Labels:

Suggested Assignees:

Affected Product Version:

OS, DB, other environment details and versions:

Steps to reproduce:

Related Issues:

Alert subscription through admin portal should send emails for all APIs

Description:

Currently, email alert subscription through admin portal will send emails for only the APIs which the admin user created. We need to improve this to support sending email alerts for all APIs when subscribed through admin portal.

Suggested Labels:

Suggested Assignees:

Affected Product Version:
2.1.0
OS, DB, other environment details and versions:

Steps to reproduce:

Related Issues:

TierCorssing Alert Email is not sent if the endUser of the API call is not the subscriber of the API

Description:
TierCorssing email alert is checking the following condition before sending an email.

from tierLimitHittingAlert#window.length(1) join stakeholderInfoTable
on (tierLimitHittingAlert.subscriber == stakeholderInfoTable.userId and true == stakeholderInfoTable.isSubscriber )

In this case, if only the application owner throttles out the tier an email will be sent. If an end user who is not the application owner throttles out the tier, then email should be sent to the email addresses specified by the subscriber.

We should be rather checking

tierLimitHittingAlert.ApplicationOwner == stakeholderInfoTable.userId

Suggested Labels:

Suggested Assignees:

Affected Product Version:

OS, DB, other environment details and versions:

Steps to reproduce:

Related Issues:

AbnormalTierUSage Email notification condition checking is wrong

Description:

Instead of applicationOwner, applicationId is compared when sending email notification. Need to fix this.

Suggested Labels:

Suggested Assignees:

Affected Product Version:
2.1.0,2.2.0
OS, DB, other environment details and versions:

Steps to reproduce:

Related Issues:

Generated gadets does not work with real time source

Description:
Gadget is created with gadget wizard using Real time data source , does not work after sending real time data.

Affected Product Version:
APIM analytics 2.1.0

Steps to reproduce:

  • Create a gadget using gadget generation wizard using Realtime data source and drag drop it to a dashboard.
  • send events to specific stream

[Alert][2.1.0] One of the alert triggering cause for HealthAvailability alerts is not working properly

Description:

There are 3 causes to trigger health availability alert.
When you look at triggering cause where the request count per minute is normal, but the response count per minute is low, the relevant Siddhi query seems to be not working properly. Hence the alert generation is not happening under the correct condition.

Suggested Labels:

Alerts, HealthAvailability
Suggested Assignees:

Affected Product Version:
2.1.0, 2.0.0
OS, DB, other environment details and versions:

Steps to reproduce:
Send 10 requests for a particular API.
Execute spark script to generate relevant percentile values and update the 'ORG_WSO2_ANALYTICS_APIM_RESPONSEPERAPIPERCENTILE' table.
Check the responsesPerMinPerApiLowerPercentile value in that table ( suppose it is 8 ).
But when the alert is generating, the value for responsesPerMinLowerPercentile is taken as 1. not 8.
There is a condition which sets this value.

ifThenElse(responsesPercentileStreamWindow.responsesPerMinLowerPercentile is null, 1.0, responsesPercentileStreamWindow.responsesPerMinLowerPercentile) as responsesPerMinLowerPercentile

What happen here is if responsesPercentileStreamWindow.responsesPerMinLowerPercentile is null, then 1.0 will be responsesPerMinLowerPercentile. Otherwise it should be 8.

Related Issues:

Unsubscribing alerts does not work in Admin Portal

Description:
Unsbuscribing alert in Admin portal does not work in Admin Portal. The alert entries for user is still found in the AlertStakeHolderInfo table even though the laerts are unsubscribed from the UI

Suggested Labels:

Suggested Assignees:

Affected Product Version:

OS, DB, other environment details and versions:

Steps to reproduce:

Related Issues:

Unusual IP access alert generating issue.

Description:

There are a few flows in the execution plan related to unusual IP access alert script.

  1. 30 days time period is defined as 302460*60 which is not correct.
  2. When updating last access time, there is a check associated with it. it would not be necessary.
  3. Caching is enabled for IP access summary table.

Above 3 causes result in generating unnecessary alerts and it would affect the end user experience.

Suggested Labels:

unusualIPaccess, alert, 2.1.0, 2.0.0
Suggested Assignees:

Affected Product Version:
2.1.0,2.0.0
OS, DB, other environment details and versions:

Steps to reproduce:
Generate an unusual IP access alert.

  1. At first it would produce as new IP detected which is correct.
  2. If you keep sending requests, it will generate alerts (old IP is detected) continuously(which is not the expected behavior)

Related Issues:

Remove the portal application

Description:
Need to remove the portal application from the pack and UI link from management console

Suggested Labels:

Suggested Assignees:

Affected Product Version:

OS, DB, other environment details and versions:

Steps to reproduce:

Related Issues:

requestPerMinPerApiStream, responsePerMinPerApiStream has primary keys

Description:
The above mentioned streams have primary keys. This shouldn't be the case. These tables are supposed to grow with data. But since the primary keys are defined, records are updated instead of being added.

Suggested Labels:

Suggested Assignees:

Affected Product Version:

OS, DB, other environment details and versions:

Steps to reproduce:

Related Issues:

Wrong year in license version

Description:
Second line of file LICENSE should be reverted back to Version 2.0, January 2004. It looks like typo or unintended change.

Suggested Labels:
Priority/Low, Typo

Affected Versions:
eeab96f-latest (24188b4)

No identity-anonymization-tool configuration in APIM analytics.

Description:
In order to test gdpr related scenarios, we couldn't find any Identity-anonymization-tool configuration in APIM analytics pack

Suggested Labels:
Type/Bug
Severity/Blocker
Priority/Highest
Affected/APIM-analytics update 9

Affected Product Version:
APIM-analytics 2.1.0 update 9

OS, DB, other environment details and versions:
APIM-analytics 2.1.0 update 9 fresh pack

Alert suppression for some alerts is done via a history table. Need to move this to a time window

Description:
Some of the alerts are suppressed via a history table. Some have been done via a time window. Time window solution seems to be a better choice.

In future, we need to try to move the history table implementations to a time window.

Suggested Labels:

Suggested Assignees:

Affected Product Version:
2.1.0
OS, DB, other environment details and versions:

Steps to reproduce:

Related Issues:

[Alerts] [HealthAvailability] Streams perMinuteRequests and perMinuteResponses gets events with a wrong timeout window

Description:
In APIMAnalytics-RequestSummarizer-RequestSummarizer-realtime1.siddhiql, Streams perMinuteRequests and perMinuteResponses receives events with a uniqueExternalTimeBatch window which has lesser timeout value than the window.

This occurs when events are inserted into perMinuteRequestsTemp stream from perSecondRequestsWithUniqueId stream. Time Window is defined as 60 seconds but timeout value is 10 seconds. So events are flushed into perMinuteRequests stream multiple times within 60 seconds. So time window of 60 seconds gets neglected.
[This occurs similarly when events are inserted into perMinuteResponsesTemp stream from perSecondResponsesWithUniqueId stream ]

This bug causes the health availability alert's request/response count percentiles related triggering cause. Pattern count can gets incremented more than once in each minute and so alerts get triggered by lesser number of occurrences, than expected.

Affected Product Version:
2.0.0, 2.1.0, 2.2.0

How to purge data in APIM Analytics

Description:
Current documentation only provides the information for purging process data tables, but some users do not require data to be retained beyond a certain point. In that scenario, it is better to give purging information about all the tables including the tables created for stream persistence.

Suggested Labels:
apim-analytics

Affected Product Version:
apim-analytics-2.x.x

Test migration from 2.1.0 to 2.2.0

Description:
We have a slight modification in schema when moving to 2.2.0. We need to test whether the migration is working fine or whether we need to provide db migrtion scripts.

Suggested Labels:

Suggested Assignees:

Affected Product Version:

OS, DB, other environment details and versions:

Steps to reproduce:

Related Issues:

Hide Abnormal Token Refresh alert from all places until implemented

Description:
AbnormalTokenRefresh alert is not yet implemented. But we can see that through the UIs. We need to remove this until the feature is complete.

Suggested Labels:

Suggested Assignees:

Affected Product Version:
2.1.0
OS, DB, other environment details and versions:

Steps to reproduce:

Related Issues:

[2.0.0] Configured parameters in 'Health Availability' Siddhi file does not reflect in alert

Description:

Even though you configured parameters (request count, responsePerMinPattern etc), these values are not reflected in particular alerts.
Suggested Labels:

Suggested Assignees:

Affected Product Version:

OS, DB, other environment details and versions:

Steps to reproduce:
Configue APIM 2.0.0 with analytics.
Generate an API to get alert.(with backend response 500)
Configured the parameter value for different value other than default value.
Generate an alert.
You will see the alert contains the default value even though the alert was generated with the updated value.

Related Issues:

[2.0.0] API Health Availability Pattern Does Not Break On 200

Description:

If response code 500 requests were invoked consecutively more than 5 times, you get alerts. But when 200 response came and then another request which returns 500 sent, it will send an alert. Ideally, when 200 response came for a request, the next alert should be come after 5 consecutive response code 500 requests.
Suggested Labels:

Alerts, healthAvailability
Suggested Assignees:

Affected Product Version:
2.0.0
OS, DB, other environment details and versions:

Steps to reproduce:
Configure alerts with APIM 2.0.0
Make an API with 200 response code and 500 response code backends.
Invoke 500 response endpoint consecutively for 5 times, you will get an alert.
Invoke 200 response code returning endpoint.
Then again invoke 500 response code returning endpoint.
You will get an alert.

Related Issues:

Resource level latency graphs

Description:

Currently, we only have per API latency graphs. As an improvement, we need to support resource-level latency graphs.

This will require changes in Spark scripts in apim-analytics and changes in UI in publisher portal of APIM.

Affected Product Version:
2.2.0

Event Simulator in carbon console doesn't seem to work

Description:

Simulating event using the event simulator is not working.
Suggested Labels:

Suggested Assignees:

Affected Product Version:
2.1.0
OS, DB, other environment details and versions:

Steps to reproduce:

Related Issues:

Forget me tool is not working as expected

Description:
Backend gives a null pointer when running the forget me tool in 2.2.0 pack

Suggested Labels:
Type/Bug
Severity/Blocker
Priority/Highest
Affected/latest apim analytics

Affected Product Version:
Latest APIM analytics pack build from jenkins

OS, DB, other environment details and versions:
OS-Mac high sierra
statDB-Mysql
otherDB-H2
JDK-1.8.0_52
Latest APIM analytics jenkins build(16/03/2018)
APIM latest pack (16/03/2018)

Steps to reproduce:
Configure analytics and APIM as described in [1].
create and invoke an API.
Configure forgetme tool required configuration as stated in [2]
Ran the forgetme tool for subscribed user.
Backend gives following error.

[1] https://docs.wso2.com/display/AM2xx/Configuring+APIM+Analytics
[2] https://docs.wso2.com/display/AM2xx/Removing+References+to+Deleted+User+Identities

screen shot 2018-03-16 at 4 33 47 pm

screen shot 2018-03-16 at 4 34 08 pm

screen shot 2018-03-16 at 4 36 39 pm

Results:
Forget me tool throws the following error.
2018-03-16 16:21:35 INFO ForgetMeTool:168 - Generating pseudonym as pseudo name is not provided : 53b5cd5b-faf1-42d6-8b46-2dea1d4d7e30 2018-03-16 16:21:37 INFO IndexNodeCoordinator:634 - My Analytics Node ID: 86ad0d14-b634-4c4c-b4bc-fb6ee2fa5738 2018-03-16 16:21:37 INFO IndexNodeCoordinator:794 - Indexing Initialized: STANDALONE | Current Node Indexing: No 2018-03-16 16:21:37 INFO AnalyticsDataServiceImpl:228 - Current Node Data Purging: Yes 2018-03-16 16:21:37 ERROR ForgetMeExecutionEngine:112 - Interrupted while executing the processor thread : java.util.concurrent.FutureTask@6295d394 java.util.concurrent.ExecutionException: java.lang.NullPointerException at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:206) at org.wso2.carbon.privacy.forgetme.ForgetMeExecutionEngine.waitForCompletion(ForgetMeExecutionEngine.java:109) at org.wso2.carbon.privacy.forgetme.ForgetMeExecutionEngine.execute(ForgetMeExecutionEngine.java:91) at org.wso2.carbon.privacy.forgetme.ForgetMeTool.process(ForgetMeTool.java:189) at org.wso2.carbon.privacy.forgetme.ForgetMeTool.main(ForgetMeTool.java:129) Caused by: java.lang.NullPointerException at org.wso2.carbon.privacy.forgetme.analytics.streams.instructions.AnalyticsStreamsInstruction.filterRecords(AnalyticsStreamsInstruction.java:146) at org.wso2.carbon.privacy.forgetme.analytics.streams.instructions.AnalyticsStreamsInstruction.execute(AnalyticsStreamsInstruction.java:93) at org.wso2.carbon.privacy.forgetme.ForgetMeExecutionEngine$ProcessorPipeline.call(ForgetMeExecutionEngine.java:216) at org.wso2.carbon.privacy.forgetme.ForgetMeExecutionEngine$ProcessorPipeline.call(ForgetMeExecutionEngine.java:187) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2018-03-16 16:21:37 INFO ForgetMeExecutionEngine:115 - All processors have been properly shut-down

Some alerts are checking for thew wrong condition to send email alerts

Description:

abnormalRequestsPerMinAlert
requestPatternChangedStream
unusualIPAccess
abnormalTokenRefresh
abnormalTierUsageAlert

has to check whether the applicationOwner is same as the stakeHolderInfo.userId

Suggested Labels:

Suggested Assignees:

Affected Product Version:

OS, DB, other environment details and versions:

Steps to reproduce:

Related Issues:

2.0.0 startup error

When i started it, i found below error in the console and wso2carbon.log

TID: [-1234] [] [2016-08-05 15:00:53,291]  INFO {org.wso2.carbon.ntask.core.impl.AbstractQuartzTaskManager} -  Task scheduled: [-1234][ANALYTICS_SPARK][APIM_LAST_ACCESS_TIME_SCRIPT] {org.wso2.carbon.ntask.core.impl.AbstractQuartzTaskManager}
TID: [-1234] [] [2016-08-05 15:00:53,295] ERROR {org.wso2.carbon.application.deployer.internal.ApplicationManager} -  Error while deploying stored cApp : org.wso2.carbon.application.deployer.internal.ApplicationManager$PendingApplication@4f756ca1 {org.wso2.carbon.application.deployer.internal.ApplicationManager}
java.lang.RuntimeException: Invalid input path generated, the input cannot contain backtracking path elements
    at org.wso2.carbon.analytics.datasource.core.util.GenericUtils.checkAndReturnPath(GenericUtils.java:787)
    at org.wso2.carbon.analytics.dataservice.core.AnalyticsDataPurgingDeployer.registerPurgingTasks(AnalyticsDataPurgingDeployer.java:147)
    at org.wso2.carbon.analytics.dataservice.core.AnalyticsDataPurgingDeployer.deployArtifacts(AnalyticsDataPurgingDeployer.java:79)
    at org.wso2.carbon.application.deployer.internal.ApplicationManager.deployCarbonApp(ApplicationManager.java:263)
    at org.wso2.carbon.application.deployer.internal.ApplicationManager.tryDeployPendingCarbonApps(ApplicationManager.java:131)
    at org.wso2.carbon.application.deployer.internal.ApplicationManager.registerDeploymentHandler(ApplicationManager.java:122)
    at org.wso2.carbon.application.deployer.internal.AppDeployerServiceComponent.setAppHandler(AppDeployerServiceComponent.java:153)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.eclipse.equinox.internal.ds.model.ComponentReference.bind(ComponentReference.java:376)
    at org.eclipse.equinox.internal.ds.model.ServiceComponentProp.bindReference(ServiceComponentProp.java:444)
    at org.eclipse.equinox.internal.ds.InstanceProcess.dynamicBind(InstanceProcess.java:416)
    at org.eclipse.equinox.internal.ds.Resolver.getEligible(Resolver.java:334)
    at org.eclipse.equinox.internal.ds.SCRManager.serviceChanged(SCRManager.java:222)
    at org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:107)
    at org.eclipse.osgi.framework.internal.core.BundleContextImpl.dispatchEvent(BundleContextImpl.java:861)
    at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
    at org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:148)
    at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:819)
    at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:771)
    at org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.register(ServiceRegistrationImpl.java:130)
    at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.registerService(ServiceRegistry.java:214)
    at org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:433)
    at org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:451)
    at org.wso2.carbon.analytics.dataservice.core.AnalyticsDataServiceComponent.activate(AnalyticsDataServiceComponent.java:74)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.eclipse.equinox.internal.ds.model.ServiceComponent.activate(ServiceComponent.java:260)
    at org.eclipse.equinox.internal.ds.model.ServiceComponentProp.activate(ServiceComponentProp.java:146)
    at org.eclipse.equinox.internal.ds.model.ServiceComponentProp.build(ServiceComponentProp.java:345)
    at org.eclipse.equinox.internal.ds.InstanceProcess.buildComponent(InstanceProcess.java:620)
    at org.eclipse.equinox.internal.ds.InstanceProcess.buildComponents(InstanceProcess.java:197)
    at org.eclipse.equinox.internal.ds.Resolver.getEligible(Resolver.java:343)
    at org.eclipse.equinox.internal.ds.SCRManager.serviceChanged(SCRManager.java:222)
    at org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:107)
    at org.eclipse.osgi.framework.internal.core.BundleContextImpl.dispatchEvent(BundleContextImpl.java:861)
    at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
    at org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:148)
    at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:819)
    at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:771)
    at org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.register(ServiceRegistrationImpl.java:130)
    at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.registerService(ServiceRegistry.java:214)
    at org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:433)
    at org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:451)
    at org.wso2.carbon.ntask.core.internal.TasksDSComponent.activate(TasksDSComponent.java:106)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.eclipse.equinox.internal.ds.model.ServiceComponent.activate(ServiceComponent.java:260)
    at org.eclipse.equinox.internal.ds.model.ServiceComponentProp.activate(ServiceComponentProp.java:146)
    at org.eclipse.equinox.internal.ds.model.ServiceComponentProp.build(ServiceComponentProp.java:345)
    at org.eclipse.equinox.internal.ds.InstanceProcess.buildComponent(InstanceProcess.java:620)
    at org.eclipse.equinox.internal.ds.InstanceProcess.buildComponents(InstanceProcess.java:197)
    at org.eclipse.equinox.internal.ds.Resolver.getEligible(Resolver.java:343)
    at org.eclipse.equinox.internal.ds.SCRManager.serviceChanged(SCRManager.java:222)
    at org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:107)
    at org.eclipse.osgi.framework.internal.core.BundleContextImpl.dispatchEvent(BundleContextImpl.java:861)
    at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
    at org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:148)
    at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:819)
    at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:771)
    at org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.register(ServiceRegistrationImpl.java:130)
    at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.registerService(ServiceRegistry.java:214)
    at org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:433)
    at org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:451)
    at org.wso2.carbon.core.internal.StartupFinalizerServiceComponent.completeInitialization(StartupFinalizerServiceComponent.java:199)
    at org.wso2.carbon.core.internal.StartupFinalizerServiceComponent.serviceChanged(StartupFinalizerServiceComponent.java:288)
    at org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:107)
    at org.eclipse.osgi.framework.internal.core.BundleContextImpl.dispatchEvent(BundleContextImpl.java:861)
    at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
    at org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:148)
    at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:819)
    at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:771)
    at org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.register(ServiceRegistrationImpl.java:130)
    at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.registerService(ServiceRegistry.java:214)
    at org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:433)
    at org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:451)
    at org.wso2.carbon.server.admin.internal.ServerAdminServiceComponent.activate(ServerAdminServiceComponent.java:106)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.eclipse.equinox.internal.ds.model.ServiceComponent.activate(ServiceComponent.java:260)
    at org.eclipse.equinox.internal.ds.model.ServiceComponentProp.activate(ServiceComponentProp.java:146)
    at org.eclipse.equinox.internal.ds.model.ServiceComponentProp.build(ServiceComponentProp.java:345)
    at org.eclipse.equinox.internal.ds.InstanceProcess.buildComponent(InstanceProcess.java:620)
    at org.eclipse.equinox.internal.ds.InstanceProcess.buildComponents(InstanceProcess.java:197)
    at org.eclipse.equinox.internal.ds.Resolver.getEligible(Resolver.java:343)
    at org.eclipse.equinox.internal.ds.SCRManager.serviceChanged(SCRManager.java:222)
    at org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:107)
    at org.eclipse.osgi.framework.internal.core.BundleContextImpl.dispatchEvent(BundleContextImpl.java:861)
    at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
    at org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:148)
    at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:819)
    at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:771)
    at org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.register(ServiceRegistrationImpl.java:130)
    at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.registerService(ServiceRegistry.java:214)
    at org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:433)
    at org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:451)
    at org.wso2.carbon.core.init.CarbonServerManager.initializeCarbon(CarbonServerManager.java:514)
    at org.wso2.carbon.core.init.CarbonServerManager.start(CarbonServerManager.java:219)
    at org.wso2.carbon.core.internal.CarbonCoreServiceComponent.activate(CarbonCoreServiceComponent.java:94)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.eclipse.equinox.internal.ds.model.ServiceComponent.activate(ServiceComponent.java:260)
    at org.eclipse.equinox.internal.ds.model.ServiceComponentProp.activate(ServiceComponentProp.java:146)
    at org.eclipse.equinox.internal.ds.model.ServiceComponentProp.build(ServiceComponentProp.java:345)
    at org.eclipse.equinox.internal.ds.InstanceProcess.buildComponent(InstanceProcess.java:620)
    at org.eclipse.equinox.internal.ds.InstanceProcess.buildComponents(InstanceProcess.java:197)
    at org.eclipse.equinox.internal.ds.Resolver.getEligible(Resolver.java:343)
    at org.eclipse.equinox.internal.ds.SCRManager.serviceChanged(SCRManager.java:222)
    at org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:107)
    at org.eclipse.osgi.framework.internal.core.BundleContextImpl.dispatchEvent(BundleContextImpl.java:861)
    at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
    at org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:148)
    at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:819)
    at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:771)
    at org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.register(ServiceRegistrationImpl.java:130)
    at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.registerService(ServiceRegistry.java:214)
    at org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:433)
    at org.eclipse.equinox.http.servlet.internal.Activator.registerHttpService(Activator.java:81)
    at org.eclipse.equinox.http.servlet.internal.Activator.addProxyServlet(Activator.java:60)
    at org.eclipse.equinox.http.servlet.internal.ProxyServlet.init(ProxyServlet.java:40)
    at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.init(DelegationServlet.java:38)
    at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1282)
    at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1195)
    at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1085)
    at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5318)
    at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5610)
    at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:147)
    at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1572)
    at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1562)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

[Alert][2.0.0] requestPerApiCountStream and responsePerApiCountStream gets aggregated counts

Description:
HealthAvailability alerts is not working properly because of this issue. What happens here is in 'APIMAnalytics-HealthAvailabilityPerMinAlert-HealthAvailabilityPerMin-realtime1' execution plan, below queries are wrong.

/* timeBatch used because Minutely updates are needed */
@info(name = 'query1')
from requestPerMinStream
select api_version, apiPublisher, tenantDomain, sum(total_request_count) as requestsPerMinPerApi
group by api_version, tenantDomain
insert into requestPerApiCountStream;

@info(name = 'query2')
from responseSummaryStream
select api_version, tenantDomain, sum(total_response_count) as responsesPerMinPerApi
group by api_version, tenantDomain
insert into responsePerApiCountStream;

If consider the 1st query, requestsPerMinPerApi value gets aggregated after each minute. That is wrong.

Suggested Labels:
Alerts, HealthAvailability

Affected Product Version:
2.0.0

Fix suggesting:

Removing getting the sum as below

/* timeBatch used because Minutely updates are needed */
@info(name = 'query1')
from requestPerMinStream
select api_version, apiPublisher, tenantDomain, total_request_count as requestsPerMinPerApi
group by api_version, tenantDomain
insert into requestPerApiCountStream;

@info(name = 'query2')
from responseSummaryStream
select api_version, tenantDomain, total_response_count as responsesPerMinPerApi
group by api_version, tenantDomain
insert into responsePerApiCountStream;

Issues when creating custom gadgets

I'm unable to create custom gadgets:

  • For bar chart,I'm getting the following error:
    Uncaught Error: Unrecognized scale type: int(11)
    -For donut chart,I'm getting a new extra item 'undefined' in the legend

Other questions please, how to support new types,for ex double bar chart?

Outdated links in the APIM Analytics 2.1.0 Management Console

Description:
There are four outdated links in the APIM Analytics Management Console (https://:9444/carbon) sign in page, as mentioned below.

  1. User Guide - Change this link to https://docs.wso2.com/display/AM210/Analytics
  2. Issue Tracker - Link this to https://github.com/wso2/analytics-apim/issues/
  3. Docs - Change this link to https://docs.wso2.com/display/AM210/Analytics
  4. About - Remove this link. (I already had a discussion with NuwanD in this regard and he confirmed my suggestion).

Suggested Assignees:
Chanaka Jayasena

Affected Product Version:
APIM Analytics 2.1.0

OS, DB, other environment details and versions:

Steps to reproduce:
Start API-M 2.1.0
Access https://:9444/carbon

Related Issues:
N/A

Outdated links in the APIM Analytics 2.2.0 Management Console

Description:
There are four outdated links in the APIM Analytics Management Console (https://:9444/carbon) sign in page, as mentioned below and highlighted in the screenshot.

  1. User Guide - Change this link to https://docs.wso2.com/display/AM220/Analytics
  2. Issue Tracker - Link this to https://github.com/wso2/analytics-apim/issues/
  3. Docs - Change this link to https://docs.wso2.com/display/AM220/Analytics
  4. About - Remove this link. (I already had a discussion with NuwanD in this regard and he confirmed my suggestion).

screen shot 2018-05-18 at 1 58 29 pm

Suggested Assignees:
Chanaka Jayasena

Affected Product Version:
APIM Analytics 2.2.0

OS, DB, other environment details and versions:

Steps to reproduce:
Start API-M 2.2.0
Access https://:9444/carbon

Related Issues:
N/A

percentile values for request count becoming negative

Description:

requestsPerDayPercentile value becoming negative in ORG_WSO2_ANALYTICS_APIM_PERCENTILE_TBL.

aau

Suggested Labels:

Suggested Assignees:

Affected Product Version:

OS, DB, other environment details and versions:

Steps to reproduce:

Related Issues:

Analytics Release for 2.2.0

New analytics release is needed for APIM 2.2.0, with new features and bug fixes from Data Analytics Server

AbnormalTokenRefresh Alert is not tested and not in documentation

Description:
We haven't documented abnormal token refresh alert. Implementation artifacts are already available. We need to test this and document it.
Suggested Labels:

Suggested Assignees:

Affected Product Version:

OS, DB, other environment details and versions:

Steps to reproduce:

Related Issues:

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.