magento / inventory Goto Github PK
View Code? Open in Web Editor NEWMagento Inventory Project (a.k.a MSI)
License: Open Software License 3.0
Magento Inventory Project (a.k.a MSI)
License: Open Software License 3.0
[HLD] Inconsistent saving of Stock Data
We have the following situation in _\Magento\Catalog\Model\Product.php#L2298
/**
* Check whether stock status changed
*
* @return bool
*/
private function isStockStatusChanged()
{
$stockItem = null;
$extendedAttributes = $this->getExtensionAttributes();
if ($extendedAttributes !== null) {
$stockItem = $extendedAttributes->getStockItem();
}
$stockData = $this->getStockData();
return (
(is_array($stockData))
&& array_key_exists('is_in_stock', $stockData)
&& (null !== $stockItem)
&& ($stockItem->getIsInStock() != $stockData['is_in_stock'])
);
}
We need to DELETE or REFACT this code.
So, we need to check the origin of the Stock Status value. We can solve in several ways:
class ProductOriginStockStatusChecker
{
public function getValue()
{
// resolving from origin data or direct call to db or
}
}
[HLD] Inconsistent saving of Stock Data
The logic of the following classes:
PAY ATTENTION: this logic is needed ONLY for BI (saving stock item data on product save). In proper way, we need to save StockItem only via StockItemRepositoryInterface
Also, we now save via stockregistry only for BI.
/**
* Dockblock like as "this logic is needed ONLY for BI (saving stock item data on product save). In propel way, we need to save StockItem only via StockItemRepository"
*/
class SaveInventoryProcessor (NOT INTERFACE)
{
/**
* Saving product inventory data
*
* @param Product $product
* @return $this
* @throws LocalizedException
*/
public function execute(Product $product)
{
$stockItem = $this->getStockItemToBeUpdated($product);
if (null !== $stockItem) {
$stockItem->setProductId($product->getId());
$stockItem->setWebsiteId($this->stockConfiguration->getDefaultScopeId());
$this->stockItemValidator->validate($product, $stockItem);
$this->stockRegistry->updateStockItemBySku($product->getSku(), $stockItem);
}
return $this;
}
/**
* Return the stock item that needs to be updated
*
* @param ProductInterface $product
* @return StockItemInterface|null
*/
private function getStockItemToBeUpdated($product)
{
$stockItem = null;
$extendedAttributes = $product->getExtensionAttributes();
if ($extendedAttributes !== null) {
$stockItem = $extendedAttributes->getStockItem();
}
if ($stockItem === null) {
$stockItem = $this->stockRegistry->getStockItem($product->getId());
}
return $stockItem;
}
}
The processing logic of saving data via model and via product repository must be the same!
For example:
<?php
/**
* Copyright © 2016 Magento. All rights reserved.
* See COPYING.txt for license details.
*/
namespace Magento\CatalogInventory\Model;
use Magento\Catalog\Api\Data\ProductInterface;
use Magento\CatalogInventory\Api\Data\StockItemInterface;
use Magento\CatalogInventory\Api\StockConfigurationInterface;
use Magento\CatalogInventory\Api\StockRegistryInterface;
use Magento\Framework\Exception\LocalizedException;
/**
* StockItemValidator
*/
class StockItemValidator (NOT INTERFACE)
{
/**
* @var StockConfigurationInterface
*/
private $stockConfiguration;
/**
* @var StockRegistryInterface
*/
private $stockRegistry;
/**
* @param StockConfigurationInterface $stockConfiguration
* @param StockRegistryInterface $stockRegistry
*/
public function _construct(
StockConfigurationInterface $stockConfiguration,
StockRegistryInterface $stockRegistry
) {
$this->stockConfiguration = $stockConfiguration;
$this->stockRegistry = $stockRegistry;
}
/**
* @param ProductInterface $product
* @param StockItemInterface $stockItem
* @throws LocalizedException
* @return void
*/
public function validate(ProductInterface $product, StockItemInterface $stockItem)
{
$defaultScopeId = $this->stockConfiguration->getDefaultScopeId();
$defaultStockId = (int)$this->stockRegistry->getStock($defaultScopeId)->getStockId();
$stockId = (int)$stockItem->getStockId();
if ($stockId !== null && $stockId !== $defaultStockId) {
throw new LocalizedException(
__('Invalid stock id: %1. Only default stock with id %2 allowed', $stockId, $defaultStockId)
);
}
$stockItemId = $stockItem->getItemId();
if ($stockItemId !== null && (!is_numeric($stockItemId) || $stockItemId <= 0)) {
throw new LocalizedException(
__('Invalid stock item id: %1. Should be null or numeric value greater than 0', $stockItemId)
);
}
$defaultStockItemId = $this->stockRegistry->getStockItem($product->getId())->getItemId();
if ($defaultStockItemId && $stockItemId !== null && $defaultStockItemId !== $stockItemId) {
throw new LocalizedException(
__('Invalid stock item id: %1. Assigned stock item id is %2', $stockItemId, $defaultStockItemId)
);
}
}
}
Also need to investigate about moving this logic in StockItemRepository
I just updated from Magento 2.1.7 to 2.1.8 and I noticed when I log in to my dashboard and click on the marketing tab, I see two widget list links and both of them go to a blank page. Can you please tell me if this is a bug?
As a Merchant I would like to be able create and manage inventory locations so that my sources can be used for delivery and inventory management with the correct information assigned to each source.
Acceptance Criteria
Documentation:
Api-functionality tests for web api
Integration tests for controllers
Description
Define Source API interfaces
Module name: InventoryApi
Edition: Community Edition
Acceptance Criteria
Documentation:
Currently, we have a Source toggling. To support Disabling the whole source (thus, all the SourceItems allocated in this Source became available for Stock aggregation and should not be taken into account for all Inventory calculations).
Need to add Disable/Enable Button on the Source edit page in admin panel.
Click on that button leads to Enabling/Disabling Source
For now, we have SourceRepositoryInterface which could be used for these purposes (save method updates ENABLE attribute of Source entity).
But we would like to introduce new service for these purposes (SourceDisabling).
*In case of any questions - send an email to [email protected], or ask a question in #MSI Slack channel
Provide Technical Design for StockItem Index
Currently, Inventory module installed with an absence of Sources. It's incorrect because merchant needs to have some "Default" Source to deal with, as it's implemented in Single Stock Magento.
Read this document for more details - https://github.com/magento-engcom/msi/wiki/The-Concept-of-Default-Source-and-Domain-Driven-Design
What we need to do is:
to Add Install data with default Source created. Pre-fill data for Default Source.
to check that after Magento Installation Default Source added. Cover this behavior with Integration test.
*In case of any questions - send an email to [email protected], or ask a question in #MSI Slack channel
Per Source Selection Algorithms HLD
To support Minimal Delivery Cost algorithm, Magento should have some source of information about the shipping prices. This information can be statically stored in the database. The source of information should be imported from the CSV file.
Magento will allow creating rules of selecting the fulfillment warehouse based on following pieces of information:
The rules will be defined in the format: ``(Customer State, Carrier) -> Warehouse.''
By default, Magento will provide the way to import the rules will be configured in the CSV file.
Module name: Inventory
Edition: Community Edition
Acceptance Criteria
Need to add to Save controllers extension point for a possibility for extending data processing.
Multiple Nodes Inventory implies that the products will be stored in multiple locations. When Customer buys the product, it should be shipped from the particular location. A customer usually does not care what location product was shipped from, as soon as this is the cheapest option. For the Merchant, it is important to have minimal overhead for the inventory storage and shipping costs. An algorithm which assigns the particular inventory to the order item should take all these considerations into account and select the best possible options.
To visualize the process of the warehouse selection let's describe the following use-case. Customers create an Order to buy products A, B and C:
Product | Qty |
---|---|
A | 5 |
B | 2 |
C | 7 |
Merchant has following stock quantities in warehouses X, Y and Z
Product | Warehouse | Qty |
---|---|---|
A | X | 10 |
A | Y | 10 |
A | Z | 10 |
B | X | 1 |
B | Y | 1 |
B | Z | 1 |
C | X | 5 |
C | Y | 2 |
C | Z | 7 |
There are multiple possible options on how to fulfill the order:
As you can see there are multiple possible strategies on how to optimize order fulfillment:
The order fulfillment algorithm should pick the location of the product and use it to calculate the shipping cost.
Earlier implementations of MNI identified the problems with the advanced algorithms of the warehouse selection. For example, if the warehouse selected based on the delivery cost, for every possible combination of warehouses system should perform the API call to the carrier to retrieve the shipping cost.
It is impossible for the Magento platform to predict all the circumstances which affect inventory and shipping costs for the particular merchant. That's why instead of implementing all possible optimizations of the algorithm selection, the framework will contain the interface which is used to resolve the Source for the order item. Different implementations of the algorithm will contain
Implementations, provided by the 3-d party extensions will allow affecting the priorities of the warehouse selection depending on the particular case where Magento is used.
Magento framework will provide the default, rule-based algorithm which allows configuring the priority of warehouses depending on the source origin, shipping address and the selected carrier.
Following interface will be called by the Magento framework at the point of shipment calculation. Changing the implementation of the interface allows affecting the algorithm of the Warehouse resolution.
use Magento\Quote\Model\Quote\Address\RateRequest;
/**
* SourceShippingResolverInterface
*/
interface SourceResolverInterface
{
/**
* Resolve source shipping data
*
* @param RateRequest $request
* @param ShippingRateCalculator $shippingRateCalculator
* @return array
*/
public function resolve(RateRequest $request, ShippingRateCalculator $shippingRateCalculator);
}
Magento will allow creating rules of selecting the fulfillment warehouse based on following pieces of information:
The rules will be defined in the format: (Customer State, Carrier) -> Warehouse
The rules will be configured in the CSV file and imported to the Magento.
Following items need to be implemented in order to make Warehouse selection work in Magento:
Task for Basic implementation of StockItem Indexation
Technical Desing Could be found here - https://github.com/magento-engcom/magento2/wiki/New-Indexer-for-StockItems
The multi-source inventory affects checkout at the moment when shipping rates are calculated. Having products can be potentially shipped from the multiple locations, there will be multiple origin addresses and multiple packages which have to be shipped.
Design the extension points in the checkout process to inject the logic of multi-source selection and shipment calculation
Split Inventory module into: Inventory, InventoryCatalog
Foreign key (sku on catalog_product_entity) will be added in InventoryCatalog
[HLD] Inconsistent saving of Stock Data
/**
* {@inheritdoc}
*
* @return \Magento\Catalog\Api\Data\ProductExtensionInterface
*/
public function getExtensionAttributes()
{
$extensionAttributes = $this->_getExtensionAttributes();
if (!$extensionAttributes) {
/** @var \Magento\Catalog\Api\Data\ProductExtensionInterface $extensionAttributes */
$extensionAttributes = $this->extensionAttributesFactory->create(
\Magento\Catalog\Api\Data\ProductInterface::class
);
$this->setExtensionAttributes($extensionAttributes);
}
return $extensionAttributes;
}
After replacing, we nave a problem: the extensionAttributes is recreated every time the method is called.
The entities:
Remove the following from the \Magento\Catalog\Model\Product::afterSave:
if ($this->getStockData()) {
$this->setForceReindexEavRequired(true);
}
Adjust StockItem aggregation functionality to support OutOfStock
SourceItem during the calculation.
Currently, indexation mechanism does not take into account that some of the source items could be out of Stock and just sum up all the quantities together.
We need to fix that logic.
So, if there are 3 Sources assigned to the single Stock.
Source A: Qty - 10 Status: In Stock
Source B: Qty - 20 Status: Out Stock
Source C: Qty - 30 Status: In Stock
Current Indexation result is: 10 + 20 + 30 = 60
Expected Behaviour: 10 + 30 = 40
Changes should be covered with Unit Tests and MUST be covered by integration tests (because we can not check full re-indexation work via api-functional tests)
Fix all places in code where Quantity (both SourceItem and StockItem) is NOT float
In Database, DECIMAL type should be used (Table::TYPE_DECIMAL)
*In case of any questions - send an email to [email protected], or ask a question in #MSI Slack channel
[TBD]
[HLD] Inconsistent saving of Stock Data
The table below shows how many times the entities we're changing occur in functional tests.
First of all, it is fixtures creating.
Name | Occurrences in CE |
---|---|
stock_data | 56 |
quantity_and_stock_status | 264 |
To allow integration with the external services Web API for Source Management is required. It should provide set of endpoints enabling third-parties with CRUD operations on Sources.
SourceInterface
GET
request.GET
request.SourceInterface
data with the POST
request.SourceInterface
data with the PUT
request.Resource | Request method | Permissions | Payload | Response | Implementation | Description |
---|---|---|---|---|---|---|
/V1/inventory/source/:sourceId |
GET | Admin InventoryApi::source |
SourceInterface |
Magento\InventoryApi\Api\SourceRepositoryInterface::get |
Get Single Source by identifier | |
/V1/inventory/source/search |
GET | Admin InventoryApi::source |
SourceInterface[] |
Magento\InventoryApi\Api\SourceRepositoryInterface::getList |
Load Sources filtered by Search Criteria | |
/V1/inventory/source |
POST | Admin InventoryApi::source , InventoryApi::source_edit |
SourceInterface |
Created source_id |
Magento\InventoryApi\Api\SourceRepositoryInterface::save |
Create Source |
/V1/inventory/source/:sourceId |
PUT | Admin InventoryApi::source |
SourceInterface |
SourceInterface |
Magento\InventoryApi\Api\SourceRepositoryInterface::save |
Update Source |
Documentation: Source WebAPI
[TBD]
Introduce new Inventory index, which will produce and fill the stock_item table with StockItems based on Source to Stock assignment and Inventory on Sources.
Need to create an interface like as StockItemIndexer, StockItemManager in module Inventory
which would be used during the StockItem re-indexation to create StockItem entities
StockItem is aggregated data by Sources Qty which are assigned to the concrete Stock.
Think about how we will use this index, it will direct join, or we need to provide Api interface
Technical Design Document https://github.com/magento-engcom/magento2/wiki/New-Indexer-for-StockItems
[HLD] Inconsistent saving of Stock Data
Сreate a single point of data (source of truth) for write operations.
Proxy all methods with stock data manipulation to StockItem.
Prototype:
// \Magento\Catalog\Model\Product
class Product
...
public function __construct(
...
array $data = [],
StockRegistryInterface $stockRegistry = null,
HydratorInterface $hydrator = null
) {
...
$this->stockRegistry = $stockRegistry ?: \Magento\Framework\App\ObjectManager::getInstance()
->get(StockRegistryInterface::class);
$this->hydrator = $hydrator ?: \Magento\Framework\App\ObjectManager::getInstance()
->get(HydratorInterface::class);
}
...
/**
* {@inheritdoc}
*
* This method has been overridden only for backward compatible work with stock item (stock_data,
* quantity_and_stock_status keys)
*
* Use \Magento\Catalog\Api\Data\ProductExtensionInterface::getStockItem for retrieving of stock item data (or stock
* status index)
* Use \Magento\Catalog\Api\Data\ProductInterface for retrieving of product data
*
*/
public function getData($key = '', $index = null)
{
if ('stock_data' === $key) {
$result = $this->getStockData();
} elseif ('quantity_and_stock_status' === $key) {
$result = $this->getQuantityAndStockStatus();
} else {
$result = parent::getData($key, $index);
}
return $result;
}
/**
* {@inheritdoc}
*
* This method has been overridden only for backward compatible work with stock item (stock_data,
* quantity_and_stock_status keys)
*
* Use \Magento\Catalog\Api\Data\ProductExtensionInterface::setStockItem for updating of stock item data
* Use \Magento\Catalog\Api\Data\ProductInterface for updating of product data
*
*/
public function setData($key, $value = null)
{
if ('stock_data' === $key) {
$result = $this->setStockData($value);
} elseif ('quantity_and_stock_status' === $key && is_array($value)) {
$result = $this->setQuantityAndStockStatus($value);
} else {
$result = parent::setData($key, $value);
}
return $result;
}
/**
* @return array
* @deprecated Use \Magento\Catalog\Api\Data\ProductExtensionInterface::getStockItem for retrieving of stock item
* data (or stock status index)
*/
public function getStockData()
{
$stockItem = $this->resolveStockItem();
return $this->hydrator->extract($stockItem);
}
/**
* @param array $stockData
* @return $this
* @deprecated Use \Magento\Catalog\Api\Data\ProductExtensionInterface::setStockItem for updating of stock item data
*/
public function setStockData(array $stockData)
{
$stockItem = $this->resolveStockItem();
$this->dataObjectHelper->populateWithArray(
$stockItem,
$stockData,
StockItemInterface::class
);
return $this;
}
/**
* @return array
* @deprecated Use \Magento\Catalog\Api\Data\ProductExtensionInterface::getStockItem for retrieving of stock item
* data (or stock status index)
*/
public function getQuantityAndStockStatus()
{
return $this->getStockData();
}
/**
* @param array $quantityAndStockStatusData
* @return $this
* @deprecated Use \Magento\Catalog\Api\Data\ProductExtensionInterface::setStockItem for updating of stock item data
*/
public function setQuantityAndStockStatus(array $quantityAndStockStatusData)
{
return $this->setStockData($quantityAndStockStatusData);
}
/**
* @return StockItemInterface
*/
private function resolveStockItem()
{
$extensionAttributes = $this->getExtensionAttributes();
$stockItem = $extensionAttributes->getStockItem();
if (null === $stockItem) {
$stockItem = $this->stockRegistry->getStockItem($this->getId());
$extensionAttributes->setStockItem($stockItem);
$stockItem->setProduct($this);
}
return $stockItem;
}
Based on the description provided
https://github.com/magento-engcom/magento2/wiki/Sources-to-Sales-Channels-Mapping
we need to make a mapping between Sources and Sales Channel.
Create the UI, backend logic, and the API to manage the Source mapping to the Stock according to the HLD https://github.com/magento-engcom/magento2/wiki/Sources-to-Sales-Channels-Mapping.
AC:
Currently, StockItem index supports just "Update by Schedule" when you need to set-up Cron which will launch the Inventory reindex.
We need to support "Update on Save" as well when changes of Source inventory would be applied immediately for StockItem index
[HLD] Inconsistent saving of Stock Data
For \Magento\CatalogImportExport\Model\Import\Product\Type\AbstractType:
namespace Magento\CatalogImportExport\Model\Import\Product\Type;
abstract class AbstractType
{
/**
* This property has been added in scope of work for backward compatible of stock item processing (work with
* stock_data and quantity_and_stock_status keys)
* quantity_and_stock_status attribute will be deleted
*
* @var array
*/
private $attributesToSkip = ['quantity_and_stock_status']; // values must be configurable via DI
...
public function prepareAttributesWithDefaultValueForSave(array $rowData, $withDefaultValue = true)
{
$resultAttrs = [];
foreach ($this->_getProductAttributes($rowData) as $attrCode => $attrParams) {
if ($attrParams['is_static'] || in_array($attrCode, $this->attributesToSkip)) {
continue;
}
...
}
For the import test, \Magento\CatalogImportExport\Model\AbstractProductExportImportTestCase:
namespace Magento\CatalogImportExport\Model;
abstract class AbstractProductExportImportTestCase extends \PHPUnit_Framework_TestCase
{
...
public static $skippedAttributes = [
...
'quantity_and_stock_status',
];
...
}
[HLD] Inconsistent saving of Stock Data
Integration tests should use the only eligible API for writing data using StockItem interface
Adjust StockItem aggregation functionality to support Disabled Sources.
Currently, indexation mechanism does not take into account that some of the sources could be disabled. So that, all the source items in the scope of this Source should be inaccessible as well.
We need to fix that logic.
So, if there are 3 Sources assigned to the single Stock.
Source A: Qty - 10 Status: In Stock
Source B: Qty - 20 Status: In Stock
Source C: Qty - 30 Status: In Stock
But Source B is disabled
Current Indexation result is: 10 + 20 + 30 = 60
Expected Behaviour: 10 + 30 = 40
Changes should be covered with Unit Tests and MUST be covered by integration tests (because we can not check full re-indexation work via api-functional tests)
Merge M2 Develop Branch into MSI fork Develop to Support PHPUnit 6.1
*In case of any questions - send an email to [email protected], or ask a question in #MSI Slack channel
Per Source Selection Algorithms HLD
Delivery to fulfill the order could be made from one or several (more than one) sources
Add possibility To Delete Stocks in Admin Panel
*In case of any questions - send an email to [email protected], or ask a question in #MSI Slack channel
Per Source Selection Algorithms HLD
Following interface will be called by the Magento framework at the point of shipment calculation. Changing the implementation of the interface allows affecting the algorithm of the Warehouse resolution.
use Magento\Quote\Model\Quote\Address\RateRequest;
/**
* SourceShippingResolverInterface
*/
interface SourceResolverInterface
{
/**
* Resolve source shipping data
*
* @param RateRequest $request
* @param ShippingRateCalculator $shippingRateCalculator
* @return array
*/
public function resolve(RateRequest $request, ShippingRateCalculator $shippingRateCalculator);
}
Description
Implement Source API interfaces
Module name: Inventory
Edition: Community Edition
Acceptance Criteria
Documentation:
Update old style to new semantic style
Provide Basic Implementation for Reservation Story according to Technical Design
https://github.com/magento-engcom/magento2/wiki/Reservations
Description
Module name: Inventory
[HLD] Inconsistent saving of Stock Data
Deprecate the \Magento\Catalog\Model\Product\Attribute\Backend\Stock attribute backend model.
Remove all logic.
Taking into account proposed changes, it makes no sense to keep this logic.
[HLD] Inconsistent saving of Stock Data
Currently, the product form sends data to the server in two keys:
The best way is to keep the data consistent with Magento WebAPI:
[
ProductInterface::EXTENSION_ATTRIBUTES_KEY => [
'stock_item' => [
StockItemInterface::QTY => 1000,
StockItemInterface::IS_IN_STOCK => true,
...
],
]
You will also need to update:
Per Source Selection Algorithms HLD
Create/Update testing scenario to check following functionality.
Source Grid displays region correctly when:
Per Source Selection Algorithms HLD
Information about selected warehouse reflected on the Order and be available during the fulfillment.
[HLD] Inconsistent saving of Stock Data
For this refactoring, you will need to:
if (!isset($postData['stock_data']['is_in_stock'])) {
$stockStatus = $parentProduct->getQuantityAndStockStatus();
$postData['stock_data']['is_in_stock'] = $stockStatus['is_in_stock'];
}
[TBD]
Per Source Selection Algorithms HLD
Magento framework should invoke the interface of the source selection algorithm created at #8 at the point of shipment calculation. Changing the implementation of the interface allows affecting the algorithm of the Warehouse resolution.
The interface is used by the Magento framework during the shipping rate calculation. Shipping rate calculation which uses the interface is:
1. compatible with bundle products
2. compatible with configurable products
3. compatible with backorders
4. compatible with multi-address shipping
Digging in the code of the Inventory module, I've found a repetition of the following DocBlock above __construct()
methods:
/**
* SourceCarrierLinkManagement constructor
*
* @param ResourceConnection $connection
*/
I add this issue as a reminder to fix the SourceCarrierLinkManagement
reference with the correct class name in which the constructor is declared.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.