Giter Club home page Giter Club logo

powerbi-visuals-textfilter's Introduction

Text Filter PowerBI Visual

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

powerbi-visuals-textfilter's People

Contributors

darthtrevino avatar dm-p avatar katua avatar microsoft-github-policy-service[bot] avatar microsoftopensource avatar mohaali avatar msftgits avatar mulyukovaidar avatar stopyoukid avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

powerbi-visuals-textfilter's Issues

No support for SQL Full Text Index in Direct Query Mode

The visual does not support FULL TEXT index type. My guess is that this is depending on the DAX-SQL translation and nothing that you can do anything about. I Create this issue anyway just to inform that the feature is missing
Using CONTAINS() instead of CHARINDEX() would have made the visual supporting FULL TEXT index types

In short, the DAX code below:
FILTER(
KEEPFILTERS(VALUES('Table'[Column])),
SEARCH("/Text to be searched for/", 'Table'[Column], 1, 0) >= 1
)

Is translated into the SQL code below:
WHERE
(
CHARINDEX(N'/Text to be search for/',
COALESCE([t10].[Column], '')
)

And CHARINDEX does not take advantage of indexes of type "FULL TEXT"

/* DAX and SQL (short versions) code below */

The DAX Code generate looks something like below:
// DAX Query
DEFINE
VAR __DS0FilterTable =
FILTER(
KEEPFILTERS(VALUES('Table'[Column])),
SEARCH("/Text to be searched for/", 'Table'[Column], 1, 0) >= 1
)

VAR __DS0Core =
CALCULATETABLE(
SUMMARIZE(
'Table',
'Table'[Column],
'Table'[Column2],
'Table'[Column3]
),
KEEPFILTERS(__DS0FilterTable)
)

VAR __DS0PrimaryWindowed =
TOPN(
501,
__DS0Core
)

EVALUATE
__DS0PrimaryWindowed

The SQL looks something like below:
// SQL
SELECT
TOP (501) [t10].[Column], [t10].[Column2], [t10].[Column3]
FROM
(
(select [$Table].[Column] as [Column],
[$Table].[Column2] as [Column2],
[$Table].[Column3] as [Column3]
from [schema].[Table] as [$Table])
)
AS [t10]
WHERE (CHARINDEX(N'/Text to be search for/', COALESCE([t10].[Column], '')) >= 1)

cannot reset the other visualizations depenedent on the text filter

After inserting the text values in the search filter, it filters values correctly in all visualizations but after clearing the text filter, it does not reset the visualizations. They stay with the same filtered values. It just clears whats written in the filter. How can I able to reset the filter?

1.2.1 stops working after a few days after upload to the Power BI Service

The filter seems to work fine in desktop and when uploaded to the Power BI service initially.

After a few days the filter stops working. When typing in the search field and pressing enter, the text filter and related table flash but the filter does not apply.

If I download the pbix, remove and re-add the linked field under the visualizations section, it works again for a couple of days

Not sure if this is related to filtering a table of over 1000 rows?

Support Wildcards

This is more of a future enhancement, but it would be nice if text filter supports wildcards. From a customer:

E.g. ERC* only finds things that begin with ERC
nnnABC begins with 3 numbers
B?B - search for bob, bib, brb, etc. any letter in between the B's

Currently, TextFilter uses applyJSONFilter, and that only really supports equals, contains and starts with operators (and their counterparts does not ) for text fields, so we cannot implement at this time.

Remove Rows limitation

Would it be possible to make use of the fetchMoreData() functionality to pull rows in segments to bypass the 30K rows limitation ?

Investigate adding filter operators to formatting pane

We could probably support adding filter operators to the formatting pane. Right now Text Filter uses the Contains operator, but there is no reason that it couldn't be a setting in the formatting pane. We could add these operators:

  • (Does not) Contains
  • (Does not) Equals
  • (Does not) Starts Width

The visual tries to read data from the data source - Negative impact on large data sources in Direct Query

I tried this filter using direct query. I haven't used it for in memory models, but for direct queries I see no point in reading data from the data source, this just put to much load on the data source.

My suggestion is to implement a settings property where developers can choose to not read data from the data source.

I get this error message while using this visual on data sources that has large amount of data in the text column

-->
Couldn't load the data for this visual
Couldn't retrieve the data for this visual. Please try again later.
Please try again later or contact support. If you contact support, please provide these details.
Activity ID: 5309b3b7-9142-473e-93ff-8b4dbb7abcd0
Request ID: c79958ce-841f-4b21-0792-502916286a85
Correlation ID: dd4f7fe6-468b-64e4-e280-dfa66d3c8e4f
Time: Thu Nov 12 2020 08:36:27 GMT+0100 (Central European Standard Time)
Service version: 13.0.14696.63
Client version: 2011.1.03669-train
Cluster URI: https://wabi-north-europe-redirect.analysis.windows.net/
<--

The error message is not due to a large number of rows, since using this visual on other columns in the same table doesn't generate this error. This error messages comes out of that this column has a large amount of data.

Yes I know that Full Text Index is not supported and using this filter on large text fields and tables will be slow, but the customers/users are willing to accept this being slow, but not that an error message/varning is generated.

By some reason, I see this error more frequently in my P5 Premium tenant than in my Power BI Desktop

Checking the generated DAX-code, there is a limit in the number of rows retrieved:
-->
DEFINE
VAR __DS0Core =
DISTINCT('Test Tickets'[Description])

VAR __DS0BodyLimited =
TOPN(30002, __DS0Core, 'Test Tickets'[Description], 1)

EVALUATE
__DS0BodyLimited

ORDER BY
'Test Tickets'[Description]
<--

But I guess that the combination of 30002 rows of NVARCHAR(4000)) in some situations is just to much for Power BI to handle, see the SQL code below:
-->
SELECT
TOP (30002) [t10].[Description]
FROM
(

SELECT
CAST([Description] AS NVARCHAR(4000))
...
<--

Problems building the visual

This is closely related to #21. Our customers can't wait so we decided to do the modifications ourself. We then have two problems. As they might be connected I only create one issue

  1. The source code is not possible to build since things are missing or wrong
  2. When fixing the problems above (at least as we think), the visual did not interact as a filter in Power BI

We start with (1), the problem to build the code

1) Build tools was already installed

2) Downloaded the latest version from GitHub
I then also changed the visual guid

3) npm install
This created the node_modules folder

4) pbiviz package
This generated this output:
info Compile C:\Users\ds82395\Downloads\PowerBI-visuals-TextFilter.tmp\precompile\visualPlugin.ts
Error parsing bundle asset "C:\Users\ds82395\Downloads\PowerBI-visuals-TextFilter.tmp\drop\visual.js": no such file
No bundles were parsed. Analyzer will show only original module sizes from stats file.
Webpack Bundle Analyzer saved report to C:\Users\ds82395\Downloads\PowerBI-visuals-TextFilter\webpack.statistics.prod.html
ERROR Failed to compile with 1 errors 13:58:45
This dependency was not found:
Globalize in ./.tmp/build/visual.js
To install it, you can run: npm install --save Globalize

5) executed: npm install --save globalize
added 2 packages, and audited 11 packages in 1s

6) pbiviz package again
info Start packaging...
warn No such file or directory: C:\Users\ds82395\Downloads\PowerBI-visuals-TextFilter\dependencies.json
error should be object
error Invalid dependencies

7) Added an empty dependencies.json file according to below
{ "dependencies" :
{}
}

8) pbiviz package again. But this time it looks ok to me and a got a package generated
info Building visual...
warn For better development experience, we strongly recommend converting the visual to es2015 modules
warn https://microsoft.github.io/PowerBI-visuals/docs/latest/how-to-guide/migrating-to-powerbi-visuals-tools-3-0
info Compile .api/v1.11.0/PowerBI-visuals.d.ts
info Compile node_modules/powerbi-models/dist/models-noexports.d.ts
info Compile node_modules/powerbi-visuals-utils-dataviewutils/lib/index.d.ts
info Compile node_modules/powerbi-visuals-utils-typeutils/lib/index.d.ts
info Compile node_modules/powerbi-visuals-utils-svgutils/lib/index.d.ts
info Compile node_modules/powerbi-visuals-utils-interactivityutils/lib/index.d.ts
info Compile src/visual.ts
info Compile C:\Users\ds82395\Downloads\PowerBI-visuals-TextFilter.tmp\precompile\visualPlugin.ts
info Start packaging...
info Package compression enabled
info Package created!
info Finish packaging
Webpack Bundle Analyzer saved report to C:\Users\ds82395\Downloads\PowerBI-visuals-TextFilter\webpack.statistics.prod.html
DONE Compiled successfully in 1765ms 14:09:46
warn Please, make sure that the visual source code matches to requirements of certification:
info Visual must use API v2.5 and above
info The project repository must:
info Include package.json and package-lock.json;
info Not include node_modules folder
info Run npm install expect no errors
info Run pbiviz package expect no errors
info The compiled package of the Custom Visual should match submitted package.
info npm audit command must not return any alerts with high or moderate level.
info The project must include Tslint from Microsoft with no overridden configuration, and this command shouldn’t return any tslint errors.
info https://www.npmjs.com/package/tslint-microsoft-contrib
info Ensure no arbitrary/dynamic code is run (bad: eval(), unsafe use of settimeout(), requestAnimationFrame(), setinterval(some function with user input).. running user input/data etc.)
info Ensure DOM is manipulated safely (bad: innerHTML, D3.html(<some user/data input>), unsanitized user input/data directly added to DOM, etc.)
info Ensure no js errors/exceptions in browser console for any input data. As test dataset please use this sample report
info Full description of certification requirements you can find in documentation:
info https://docs.microsoft.com/en-us/power-bi/power-bi-custom-visuals-certified#certification-requirements

Then we jump into issue 2, that the visual doesn't filter

So I start with the newly built filter from above
1) Imported the visual into a simple test Power BI file that already contains the official version of the text earch filter
2) Replaced the official filter with my newly built filter
But now the filter doesn't filter. I can see that it reads the maximum 30'000 rows, but it does not filter other tables in the canvas
And now I have two (since I changed the visual Guid). But it's only the original text search visual that filters. Switching to the new does not show anything

Enhance search functionality using Advanced Filter API etc

Since this visual was built, new Filter features have become available e.g. Advanced Filter API:
https://docs.microsoft.com/en-us/power-bi/developer/visuals/filter-api#the-advanced-filter-api

This visual could be greatly improved by implementing those, e.g.

  • implement "and" functionality by default e.g. if the user enters "word1 and word2". It currently matches that as one string
  • implement "or" functionality, e.g. if the user enters "word1 or word2"
  • implement the various "Condition Operators" eg DoesNotContain - probably best as a format property

Text Filter is not working in Power BI Service

Please help me, I'm desperate !!!!
I created a report with Power BI Desktop (August 2018) and published it on power bi report service, August 21st 2018.
In editing, everything works great but once I publish in report server the visual will not render.
I'm using google chrome version 88.xxx.xxxxx

What's the solution?

Search more than one data field

If the box could search more than 1 field that would be phenomenal. Otherwise, I have to have multiple visuals to search multiple fields. I'd like to be able to search multiple fields (even if it is limited to only like 3 or 4 fields) for certain words within the data.

Enhanced Text Search functionality

The existing Text Search is really good and with the "" wildcard is does support ANDing of search terms.
e.g. a search term quality
food would match strings "quality food" and "great quality and great food"

However it would be really powerful if additional search term operators could be added, such as the following examples

ORing - A logical OR of search terms so for example the terms pub OR bar OR club would match strings containing any of those terms removing the need to search for them separately

NOT - A logical NOT of a search term so for example pub NOT bar would find strings containing pub but not containing club

ANYORDER - Match ANDed terms that appear in any order in a string
e.g. ANYORDER [pub*bar] would match strings "I go to pubs and bars" and "I go to bars and pubs"

Many users are used to using advanced search operators and these types of additional operators would really add power to searching and analysing text data from sources such as social data, CX and user feedback.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.