vuestorefront / magento2-vsbridge-indexer Goto Github PK

If Divante\VsbridgeIndexerCatalog\Model::getProductUrlSuffix() returns null it will cause an issue in Divante\VsbridgeIndexerCatalog\Model::addUrlPath().

$rewrite = mb_substr($rewrite, 0, -strlen($urlSuffix));

strlen(null) will return 0 which in turn leads to mb_substr() returning an empty string which will be indexed as url_path for the product.

The field for ElasticSearch password should be type obscure and should be encrypted. It should also be excluded from the standard config dump (bin/magento app:config:dump) as it is considered sensitive data.

Hi there! In mage2vuestorefront we're setting up the product.configurable_options.label by extrapolating the value_index to exact attribute label

https://github.com/DivanteLtd/mage2vuestorefront/blob/c71788e1c918225685344657f9d6ca0b2417d801/src/adapters/magento/product.js#L486

Can we apply this to magento-2-vsbridge indexer as well?

this version won't install with Magento 2.3.1 it has a problem with the Magento 2.3.1 ES 6.7.1 version.

Field "final_price" of Configurable Producs (the parent) is always exported with value "0" and therefore sorting and filtering by price is not working as intended.
In mage2vs it is set to the price of the cheapest configurable child.

Starting with VS 1.9 the url_path is being used for the url routing (vuestorefront/vue-storefront#2446)

The new edition of M2VS is populating these fields by the customizable code snippet:
https://github.com/DivanteLtd/mage2vuestorefront/blob/349c19063264a3fd8dd68c62571b42e32a1e98e9/src/config.js#L5

We should set the product.url_path to normalized $product->getUrl() - with protocol, domain and leading / removed (so just the path without the leading slash); for category - not sure how magento does it by default but it should be just the default magento2 url path for category

According to Magento's technical guidelines, rule 2.5 states "Proxies and interceptors MUST never be explicitly requested in constructors." Now that the M2 coding standards has added this rule the phpcs linter throws an error in the constructor of RebuildEsIndexCommand when passing in each of the following proxies in __construct. This requirement is in place because Proxies are generated files and therefore should be lazy loaded.

• IndexOperations\Proxy $indexOperations,
• StoreManagerInterface\Proxy $storeManager,
• \Magento\Framework\App\State\Proxy $state,
• \Magento\Indexer\Model\Indexer\CollectionFactory\Proxy $collectionFactory

These should be injected using di.xml as mentioned in the PR linked above.

• reindex one product/category by id.

TOD

• create new command line to reindex data for one specif product
  e.g
  bin/magento vsbridge:index vsbridge_product_indexer --id 100

On Magento 2.3.0 version have reindex error:

With magento sample data

bin/magento indexer:reindex

Design Config Grid index has been rebuilt successfully in 00:00:00
Catalog Rule Product index has been rebuilt successfully in 00:00:01
Vsbridge Category Indexer index has been rebuilt successfully in 00:00:01
Vsbridge Attributes Indexer index has been rebuilt successfully in 00:00:03
PHP Fatal error:  Uncaught TypeError: Argument 2 passed to Magento\Catalog\Block\Product\ImageFactory::getLabel() must be of the type string, null given, called in /Users/dima/sites/m230/vendor/magento/module-catalog/Block/Product/ImageFactory.php on line 154 and defined in /Users/dima/sites/m230/vendor/magento/module-catalog/Block/Product/ImageFactory.php:106
Stack trace:
#0 /Users/dima/sites/m230/vendor/magento/module-catalog/Block/Product/ImageFactory.php(154): Magento\Catalog\Block\Product\ImageFactory->getLabel(Object(Magento\Catalog\Model\Product\Interceptor), NULL)
#1 /Users/dima/sites/m230/vendor/magento/module-catalog/Block/Product/ImageBuilder.php(135): Magento\Catalog\Block\Product\ImageFactory->create(Object(Magento\Catalog\Model\Product\Interceptor), 'new_products_co...', Array)
#2 /Users/dima/sites/m230/vendor/magento/module-catalog/Block/Product/AbstractProduct.php(513): Magento\Catalog\Block\Product\ImageBuilder->create(Object(Magento\Catalog\Model\Product\Interceptor), 'new_products_co...', Array)
#3 /Users/di in /Users/dima/sites/m230/vendor/magento/module-catalog/Block/Product/ImageFactory.php on line 106
    <div class="block widget block-products-list grid">
                <div class="block-content">
            <!-- new_products_content_widget_grid-->            <div class="products-grid grid">
                <ol class="product-items widget-product-grid">
                                                                <li class="product-item">                        <div class="product-item-info">
                            <a href="http://m230.test/selene-yoga-hoodie.html" class="product-item-photo">
                                

Hey guys,

I'm on Magento 2.2.6 - I switched from the mage2vs data pump to the vsbridge-indexer and noticed the data is different for configurable products "configurable_children"

The indexer doesn't include extra text attributes that the data pump did. So like individual color and size text attributes for the child do not appear.

Mode "Update by Schedule" might not work correctly in Magento Commerce

Hi,

We are unable to run the magento installer if the plugin has been added by composer. It gives the following error

[Progress: 481 / 877]
Module 'Magento_CatalogSearch':
In PatchApplier.php line 167:
No alive nodes found in your cluster

Once the plugin is disabled in app/etc/config.php then the install process will finish.

Magento have dimensions for price indexer ...

#28

How to purge this module completely?

I have tried removing the folder, removing anything named magento2-vsbridge-indexer from composer.json and composer.lock

I have done module:disable.

Was a bit fast without reading that it wasn't tested with 2.3... Now there's a lot of issues.

Edit:

Errors during compilation:
	Divante\VsbridgeIndexerCatalog\Model\ResourceModel\Product\AttributeDataProvider
		Incompatible argument type: Required type: \Divante\VsbridgeIndexerCatalog\Model\ResourceModel\entityType. Actual type: string; File: 
/opt/bitnami/apps/magento/htdocs/vendor/divante/magento2-vsbridge-indexer/src/module-vsbridge-indexer-catalog/Model/ResourceModel/Product/AttributeDataProvider.php

	Divante\VsbridgeIndexerCatalog\Model\ResourceModel\Category\AttributeDataProvider
		Incompatible argument type: Required type: \Divante\VsbridgeIndexerCatalog\Model\ResourceModel\entityType. Actual type: string; File: 
/opt/bitnami/apps/magento/htdocs/vendor/divante/magento2-vsbridge-indexer/src/module-vsbridge-indexer-catalog/Model/ResourceModel/Category/AttributeDataProvider.php

Total Errors Count: 2

In Log.php line 92:
                            
  Error during compilation

Make it compatible with mage2vuestorefront module.

Hi, have a good day!

I try to install to magento 2.3.1 and see errors related to ES version:

vuestorefront/mage2vuestorefront#75

This is a question, not an issue. Is there a planned release schedule for this module? I see there has been only one release so far, and multiple updates to the develop branch since then. I would like to use some of these updates but would rather not use dev-develop to avoid unwanted updates.

Related to: vuestorefront/vue-storefront#3090

It will be great if we can add the parent_products to the products (simple one's) to mark up the parent products (it's probably n:m relationship, correct?). Without this relationship it's very very hard to find the parent product when you get the simple one

Or maybe there is another way?

Category Filtering on VSF home page didn't seem to work with the default indexer install using category.name.keyword on default.json file in VSF

Seems like the product category needs to be changed to TYPE_KEYWORD
on magento2-vsbridge-indexer/src/module-vsbridge-indexer-catalog/Index/Mapping/Product.php

From lines 220:

  'category' => [
                'type' => 'nested',
                'properties' => [
                    'category_id' => ['type' => FieldInterface::TYPE_LONG],
                    'position' => ['type' => FieldInterface::TYPE_LONG],
                    'name' => ['type' => FieldInterface::TYPE_TEXT],
                ],
            ],

To

       'category' => [
                // 'type' => 'nested',
                'properties' => [
                    'category_id' => ['type' => FieldInterface::TYPE_LONG],
                    'position' => ['type' => FieldInterface::TYPE_LONG],
                    'name' => [
                        'type' => FieldInterface::TYPE_TEXT,
                        'fields' => [
                            'keyword' => [
                                'type' => FieldInterface::TYPE_KEYWORD,
                                'ignore_above' => 256
                            ]
                        ]
                    ]
                ]
            ],

This worked for me for vsf 1.8-1.9

It would be nice if we could somehow customize indexed attributes. E.g. I would like to fetch configurable_children.media_gallery.

Customizable Options of Simple Products are currently not exported at all. They are available in mage2vs.

I noticed when running bin/magento vsbridge:reindex --store=1, the command only runs for the specified store in Magento's context, however it still reindexes all ElasticSearch indexes for all stores.

I discovered this when trying to delete and recreate the index for each store because I had an attribute change its type and the reindex was failing due to the schema mismatch. I tried to run the command above to reindex one store to start with, and noticed all of the other stores throwing errors like this in var/log/vsbridge-indexer/info.log:

vsbridgeIndexerLogger.ERROR: Bulk index operation failed 1 times in index vue_storefront_catalog_en_us for type product. Error (mapper_parsing_exception) : failed to parse [my_attribute]. Failed doc ids sample : 1.

It does delete the index for only the specified store, as it is using the IndexOperations to run that command which accepts the store as a param. So the result is each time I run this for a store, it re-runs the index in ES for all stores, but only deletes and recreates the specified store.

I understand the challenge of resolving this as the index action $indexer->reindexAll(); calls Magento's indexer which then calls each of the indexers in these modules. At the moment I don't have any solutions to recommend, but thought I'd post it here for further discussion.

Hopefully the explanation makes sense. I feel like I kind of wrote in circles here. Let me know if you need some clarity.

https://github.com/DivanteLtd/magento2-vsbridge-indexer/blob/4702e4cbfe04537c8b61b31ee052b37971c62823/src/module-vsbridge-indexer-core/etc/config.xml#L14

+ @afirlejczyk

Unfortunately, we can't easily get the color swatches from Magento using API

@afirlejczyk can You please take a look at this and propose some solution? I mean: we should probably put the swatches into attribute metadata and use it in VS frontend (currently we're just mapping the attribute label not using any other metadata)

You should be aware that Reviews are stored in the Elasticsearch. To display Reviews correctly, you need to update your mage2vuestorefront

and run the Reviews sync:

node --harmony cli.js reviews

I setup zero quantities for all simple products includes to config product. The config product still display on catalog listing page of VSF. On magento2 Luma frontend this config not displayed. Then see ES index:

I will prepare PR.

Thanks for this awesome module!

vuestorefront/mage2vuestorefront#69

The usual setup is that we have vueSF on a separate server than Magento.
BUT composer.json (in the state how it is now) requires a specific version of elasticsearch to be installed on the MAGENTO server, then, however, we're able to configure another server host in Magento plugin settings (Magento admin)

Magento 2.3 uses ES 6.1 by default (https://devdocs.magento.com/guides/v2.3/config-guide/elasticsearch/es-overview.html) which will cause the installation to fail with:

Problem 1
    - Installation request for divante/magento2-vsbridge-indexer dev-master -> satisfiable by divante/magento2-vsbridge-indexer[dev-master].
    - Conclusion: remove elasticsearch/elasticsearch v6.1.0
    - Conclusion: don't install elasticsearch/elasticsearch v6.1.0
    - divante/magento2-vsbridge-indexer dev-master requires elasticsearch/elasticsearch ~5.1 -> satisfiable by elasticsearch/elasticsearch[v5.1.0, v5.1.1, v5.1.2, v5.1.3, v5.2.0, v5.3.0, v5.3.1, v5.3.2, v5.4.0].
    - Can only install one of: elasticsearch/elasticsearch[v5.1.0, v6.1.0].
    - Can only install one of: elasticsearch/elasticsearch[v5.1.1, v6.1.0].
    - Can only install one of: elasticsearch/elasticsearch[v5.1.2, v6.1.0].
    - Can only install one of: elasticsearch/elasticsearch[v5.1.3, v6.1.0].
    - Can only install one of: elasticsearch/elasticsearch[v5.2.0, v6.1.0].
    - Can only install one of: elasticsearch/elasticsearch[v5.3.0, v6.1.0].
    - Can only install one of: elasticsearch/elasticsearch[v5.3.1, v6.1.0].
    - Can only install one of: elasticsearch/elasticsearch[v5.3.2, v6.1.0].
    - Can only install one of: elasticsearch/elasticsearch[v5.4.0, v6.1.0].
    - Installation request for elasticsearch/elasticsearch (locked at v6.1.0) -> satisfiable by elasticsearch/elasticsearch[v6.1.0].

Solution:

• remove elasticsearch requirement from composer.json to allow the plugin to be installed regardless ES available on Magento server

Hi there,
I have installed the magento2-vsbridge-indexer. When I run the indexer:reindex for vsbridge_product_indexer

php bin/magento indexer:reindex vsbridge_product_indexer

It show the message:
Vsbridge Product Indexer indexer process unknown error:
Product 9208 is not part of the current product collection..

In case either I disabled the product(xxx) or change its visibility then it show the same message for any other product next to the previous.
Also it does not import all the products to elastic. I have approx 12K product in megento store and it import only approx 5K.
Please guide me how to fix this problem?

Thanks!

See errors on Magento 2.2.5, and ES index not created.

Related commit
cdebbbc

I have setup the vue-storefront/vue-storefront-api and magento2 server in the same instance in google cloud server. I have followed the steps written in the documentation for installing indexer in magento. I'm running vue-storefront-api and magento server in the docker container but still I'm always getting the "No alive nodes found in your cluster" error. Can anyone help me out here?

The mage2vuestorefront is importing the created_at field for products, but the indexer is not. Is there a reason for this?

My colleague created a feature that adds a "New product" badge to products added in the last n days. I couldn't understand how she had been able to get the field to elasticsearch, until I realized she was using mage2vuestorefront and I was using the indexer.

Dublicated orders in admin html, by creation via REST

Magento 2.2.6

Out of the box Search Engine using Mysql.

Expected result:
See one order in admin html

Actual result:
Multiple orders

tail -f var/log/exception

[2019-03-25 05:47:44] main.CRITICAL: Report ID: webapi-5c986b806ec6e; Message: {"error":{"root_cause":[{"type":"parsing_exception","reason":"[terms] query does not support [5082] within lookup element","line":1,"col":89}],"type":"parsing_exception","reason":"[terms] query does not support [5082] within lookup element","line":1,"col":89},"status":400} {"exception":"[object] (Exception(code: 400): Report ID: webapi-5c986b806ec6e; Message: {\"error\":{\"root_cause\":[{\"type\":\"parsing_exception\",\"reason\":\"[terms] query does not support [5082] within lookup element\",\"line\":1,\"col\":89}],\"type\":\"parsing_exception\",\"reason\":\"[terms] query does not support [5082] within lookup element\",\"line\":1,\"col\":89},\"status\":400} at /home/vuestorefront/magento2/vendor/magento/framework/Webapi/ErrorProcessor.php:205, Elasticsearch\\Common\\Exceptions\\BadRequest400Exception(code: 400): {\"error\":{\"root_cause\":[{\"type\":\"parsing_exception\",\"reason\":\"[terms] query does not support [5082] within lookup element\",\"line\":1,\"col\":89}],\"type\":\"parsing_exception\",\"reason\":\"[terms] query does not support [5082] within lookup element\",\"line\":1,\"col\":89},\"status\":400} at /home/vuestorefront/magento2/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Connections/Connection.php:610)"} []

More information: vuestorefront/vue-storefront#2577

This is more of a question than an issue but it is still an issue if you look at it from my angle @afirlejczyk . The Indices created by this extension does it include the schema needed to work with VSF-API because I have to do that manually on VSF-API or I don't get to see products in vue-storefront. If so, how do I add mappings to this extension so that when the indices are created on this extension they are fully compatible and do not need index rebuilding before they can be used.?

p.s VSF-API is vue-storefront-api

Currently you have the ability to set the index name prefix which is then appended with the Store ID for each store that is indexed. We were previously using the store code as the unique identifier for each index when working with mage2vs and we hoped to keep this functionality while using this module as well.

The advantage of using store code as opposed to ID is that it is much easier to immediately understand what that index is for. In our situation each store code is the underscore version of a locale code, so seeing vue_storefront_catalog_en_us is much easier to understand than vue_storefront_catalog_1. The more stores you have the more this becomes important, especially when working in VS configurations and such.

Add new indexer for exporting reviews to ES

Indexer is awesome, thank you for it!

But I found special case about catalog promotion.

Steps to reproduce:

1. In Magento 2 I have product with price $150
   

I see that product and price in VSF too

2. Then I set special price for product (advance price) and I see correct result in M2 front-end
   
   

and in VSF

3. Afterwards I set Catalog Price Role (50% for all products). I see correct price in M2
   

but in VSF I still see previous price

When I check Elasticsearch I see correct "final_price"

but VSF get price from "special_price" not from "final_price"

Firstly, really great work on this, indexing is really quick.

I noticed that the content types of attribute options differ from the content type provided by mage2vs. This is not an issue per se, but their is a difference i think we should discuss about.

Magento Attribute boolean types are 'true' or 'false', and importing data via mage2vs these are '1' or '0'.
Multiselect attributes with multiple options for a products are imported as arrays, that's great, but the option_ids in this array should be integers but are now strings.
Attribute options that are not filled in Magento are imported as NULL value, while in mage2vs these options are stripped from the dataset

Hi,

I'm testing this module, to see if it is ready to replace mage2vuestorefront, which we are currently using. But after installing the module and running the Magento indexers, i'm having troubles with linking the elastic indexes generated by the Magento 2 indexer with the VueStorefront API.

Error when searching for data via the API with https://my-vue-storefront.nl/api/catalog/vue_storefront_catalog/_search

{  
   "error":{  
      "root_cause":[  
         {  
            "type":"index_not_found_exception",
            "reason":"no such index",
            "resource.type":"index_or_alias",
            "resource.id":"vue_storefront_catalog",
            "index_uuid":"_na_",
            "index":"vue_storefront_catalog"
         }
      ],
      "type":"index_not_found_exception",
      "reason":"no such index",
      "resource.type":"index_or_alias",
      "resource.id":"vue_storefront_catalog",
      "index_uuid":"_na_",
      "index":"vue_storefront_catalog"
   },
   "status":404
}

Settings

I've setup the index name prefix like this:

The indexes are generated with elastic:

My vue-storefront and vue-storefront-api settings:

Are these settings allright? Or have I misinterpreted the docs?