I managed to import some attributes with backend_type = null, because in the importer I used before, varchar was assumed as default.

This causes such errors when searching in the admin panel in Catalog -> Products

 Column not found: 1054 Unknown column 'my_custom_attribute' in 'where clause' a

Do you think it would make sense to add a validation for backend_type to be set before importing?

(low prio, as the fix is to just set backend_type)

Our database uses the default utf8_general_ci on the schema, tables, and columns. The issue is on the eav_attribute_option_value table (on the "value" column).

For example for a specific attribute we have this kind of values into the import file: "Flachschaukasten" & "Flachschaukästen" When trying to import data on the sandbox, I've only one of this value (for example "Flachschaukasten") after the attribute import. Then during the product import process, there is an issue because of data "Flachschaukästen" that is required for the product but not populated in the database (We disabled the dynamic option value creation).

So we made some tests to try to find a "workaround". We put the collation utf8_bin on this column name and it's working well (at least for the import process) but we suspect having some issue in the future because of this behavior. Also, I noted that on my local I used the utf8_general_ci collation, but I am able to have both values... so I try to compare my MySQL configuration with the remote one but didn't find any difference about this kind of configuration.

Hi there,

I've a strange behavior and it seems like a bug in attribute option values.
I've a attribute.csv and just for one attribute I've set option values in default store view code and for the same attribute in one specific store view code.
In class TechDivision\Import\Attribute\Observers\AttributeOptionValueExportObserver the method $this->getLastEntityId() always returns the last entity id of previous processed attribute, but not the entity id of the current attribute for the values should be imported.
Example: I've attribute A, B, C, all attributes in default store code and in a specific store code (so 6 lines in CSV), just attribute B has option values, alll other are empty. But in echDivision\Import\Attribute\Observers\AttributeOptionValueExportObserver the function getLastEntityId() returns just the id of attribute C, but there is no entry in "ExportableTrait->getArtefactsByTypeAndEntityId" and an error will be thrown. If the execution order is A, C, B, so the attribute with the option values is the last one in csv, then it will work.

CSV:
store_view_code,attribute_set_name,attribute_group_name,entity_type_code,attribute_code,sort_order,data_type,frontend_label,backend_type,frontend_input,source_model,is_filterable,backend_model,attribute_option_values ,Default,TradeControl,catalog_product,einband,10,select,Einband,int,select,,1,, ,Default,TradeControl,catalog_product,farbe,10,select,"Farbe (ISO)",int,select,,1,, ,Default,TradeControl,catalog_product,garantie_monate,10,int,garantie_monate,int,text,,0,, ,Default,TradeControl,catalog_product,gewicht_kg,1,select,"Gewicht kg",int,select,,1,, ,Default,TradeControl,catalog_product,groesse_xxs_xxl_system,10,select,Größe,int,select,,1,, ,Default,TradeControl,catalog_product,inet,10,select,INet,int,select,,1,, ,Default,TradeControl,catalog_product,jahrgang,10,select,Jahrgang,int,select,,1,, ,Default,TradeControl,catalog_product,laender,2,multiselect,Länder,varchar,multiselect,,1,Magento\Eav\Model\Entity\Attribute\Backend\ArrayBackend, ,Default,TradeControl,catalog_product,machcode,10,select,Matchcode,int,select,,1,, ,Default,TradeControl,catalog_product,material,1,multiselect,material,varchar,multiselect,,1,Magento\Eav\Model\Entity\Attribute\Backend\ArrayBackend, ,Default,TradeControl,catalog_product,rebsorte,10,select,Rebsorte,int,select,,1,, ,Default,TradeControl,catalog_product,sonstiges,10,multiselect,sonstiges,varchar,multiselect,,1,Magento\Eav\Model\Entity\Attribute\Backend\ArrayBackend, ,Default,TradeControl,catalog_product,sprachen,2,multiselect,Sprachen,varchar,multiselect,,1,Magento\Eav\Model\Entity\Attribute\Backend\ArrayBackend,"Sprache1,Sprache2,Sprache3" ,Default,TradeControl,catalog_product,tags,9999,text,Tags,text,textarea,,0,, en,Default,TradeControl,catalog_product,einband,10,select,Cover,int,select,,1,, en,Default,TradeControl,catalog_product,farbe,10,select,"color (iso)",int,select,,1,, en,Default,TradeControl,catalog_product,garantie_monate,10,int,"warranty (month)",int,text,,0,, en,Default,TradeControl,catalog_product,gewicht_kg,1,select,"weight (kg)",int,select,,1,, en,Default,TradeControl,catalog_product,groesse_xxs_xxl_system,10,select,size,int,select,,1,, en,Default,TradeControl,catalog_product,inet,10,select,inet,int,select,,1,, en,Default,TradeControl,catalog_product,jahrgang,10,select,vintage,int,select,,1,, en,Default,TradeControl,catalog_product,laender,2,multiselect,Countries,varchar,multiselect,,1,Magento\Eav\Model\Entity\Attribute\Backend\ArrayBackend, en,Default,TradeControl,catalog_product,machcode,10,select,machcode,int,select,,1,, en,Default,TradeControl,catalog_product,material,1,multiselect,material,varchar,multiselect,,1,Magento\Eav\Model\Entity\Attribute\Backend\ArrayBackend, en,Default,TradeControl,catalog_product,sonstiges,10,multiselect,sonstiges,varchar,multiselect,,1,Magento\Eav\Model\Entity\Attribute\Backend\ArrayBackend, en,Default,TradeControl,catalog_product,sprachen,2,multiselect,Languages,varchar,multiselect,,1,Magento\Eav\Model\Entity\Attribute\Backend\ArrayBackend,"Sprache1-en,Sprache2-en,Sprache3-en" en,Default,TradeControl,catalog_product,tags,9999,text,Tags,text,textarea,,0,, en,Default,TradeControl,catalog_product,rebsorte,10,select,"Grape variety",int,select,,1,,

I was importing attributes for products, without specifying the entity_type_code

This was working fine, but when I added attribute_set_name and attribute_set_group it started failing:

+---------------------------------+------+----------------------+---------------------------------------------------------------+
| attribute-import_19700101_1.csv | 2    | attribute_set_name   | Found invalid value "Default" for column "attribute_set_name" |
| attribute-import_19700101_1.csv | 3    | attribute_group_name | Found invalid attribute group name "PIM"                      |

This validation problem can be fixed by specifying entity_type_code, but this is misleading and needed debugging.

Either we should require entity_type_code, or consistently default it to catalog_products

Here should happen a validation.

Example:

if (!isset($adminValueArtefacts[$key])) {
    // throw exception with message that points to faulty data in CSV file
}

Multiple Field Delimiter and Multiple Value Delimiter are valid configuration params, but it is undocumented.

Also the lines:

Does not use Multiple Field Delimiter value if set in configuration.
I use this fix:

$attributeOptionValues = $this->explode($this->getValue(ColumnKeys::ATTRIBUTE_OPTION_VALUES), $this->getMultipleFieldDelimiter());
$attributeOptionSortOrder = $this->explode($this->getValue(ColumnKeys::ATTRIBUTE_OPTION_SORT_ORDER), $this->getMultipleFieldDelimiter());

$attributeOptionValues = $this->explode($this->getValue(ColumnKeys::ATTRIBUTE_OPTION_VALUES), $this->getMultipleFieldDelimiter());

In magento, Visual swatch attribute can have an image or an HEX color.
Actually the importer does not handle files for visual swatch attributes.

For supporting this, I think that class OptionSubject should implement also FileUploadSubjectInterface.
The generation of swatch images is managed by the function generateSwatchVariations in Magento\Swatches\Helper\Media
I don't know if inside an Observer the use of Magento\Swatches\Helper\Media is possible in a different and more reliable way.

So the following code is just an experiment (hope this help):

/**
 * TechDivision\Import\Attribute\Observers\AttributeOptionSwatchFileUploadObserver
 */

namespace TechDivision\Import\Attribute\Observers;

use TechDivision\Import\Utils\StoreViewCodes;
use TechDivision\Import\Attribute\Utils\ColumnKeys;
use TechDivision\Import\Attribute\Utils\MemberNames;
use TechDivision\Import\Attribute\Services\AttributeBunchProcessorInterface;
use \Magento\Framework\App\Bootstrap;
use \Magento\Swatches\Helper\Media;
use \Magento\Framework\App\ObjectManager;

class AttributeOptionSwatchFileUploadObserver extends AttributeOptionSwatchUpdateObserver
{
    /** @var \Magento\Framework\App\State */
    protected $state;
		
	/** @var \Magento\Swatches\Helper\Media */
    protected $swatchHelper;
	
	/** @var \Magento\Catalog\Model\Product\Media\Config */
    protected $productMediaConfig;
	
	/**
     * The attribute processor instance.
     *
     * @var \TechDivision\Import\Attribute\Services\AttributeBunchProcessorInterface
     */
    protected $attributeBunchProcessor;
	
    private $tmpMediaPath;
    private $fullTmpMediaPath;
	
    /**
     * Initializes the observer with the passed subject instance.
     *
     * @param \TechDivision\Import\Attribute\Services\AttributeBunchProcessorInterface $attributeBunchProcessor The attribute bunch processor instance
     */
    public function __construct(AttributeBunchProcessorInterface $attributeBunchProcessor)
    {
		$this->attributeBunchProcessor = $attributeBunchProcessor;
		
		
		// adding bootstrap

		require '/path/to/your/magento/app/bootstrap.php';
		$bootstraps = Bootstrap::create(BP, $_SERVER);
		$objectManager = $bootstraps->getObjectManager();
		
		$this->state = $objectManager->get('\Magento\Framework\App\State');
		$this->state->setAreaCode('frontend');
		$this->swatchHelper = $objectManager->create('\Magento\Swatches\Helper\Media');
		
		
		$filesystem = $objectManager->create('\Magento\Framework\Filesystem');

		// Prepare visual swatches files.
		$mediaDirectory = $filesystem->getDirectoryRead(\Magento\Framework\App\Filesystem\DirectoryList::MEDIA);
		$this->productMediaConfig = $objectManager->create('\Magento\Catalog\Model\Product\Media\Config');
		
		$this->tmpMediaPath = $this->productMediaConfig->getBaseTmpMediaPath();
		$this->fullTmpMediaPath = $mediaDirectory->getAbsolutePath($this->tmpMediaPath);

    }

    /**
     * Process the observer's business logic.
     *
     * @return void
     */
    protected function process()
    {
	
        $attr = array(
                    MemberNames::OPTION_ID  => $this->getLastOptionId()
                );
		$attributeOptionSwatch = $this->initializeAttribute($attr);
		
		if (isset($attributeOptionSwatch['type']) && $attributeOptionSwatch['type'] == 2)
		{
			$swatchVisualFile = $attributeOptionSwatch['value'];
			
			$this->getSubject()->setFilesDispersion(true);
			//$this->getSubject()->setMediaDir($this->tmpMediaPath);
			
			// upload the file and set the new image path
			$imagePath = $this->getSubject()->uploadFile($swatchVisualFile);

			// log a message that the image has been copied
			$this->getSubject()->getSystemLogger()->debug(
				sprintf('Successfully copied image %s => %s', $imagePath, $imagePath)
			);
			$this->swatchHelper->generateSwatchVariations($imagePath);
			
			$attributeOptionSwatch['value'] = $imagePath;

			$this->persistAttributeOptionSwatch($attributeOptionSwatch);
		}
    }
	
}

The function setFilesDispersion set to true a new property in FileUploadTrait .
I have integrated in FileUploadTrait the same code that in Magento is used when a new image is uploaded if dispersion paths are used.
In my Magento installation, the use of dispersion paths is active for product images and swatch attributes images. Maybe we could integrate this code into FileUploadTrait and it should be managed by configuration

Magento functions:

    /**
     * Get dispertion path
     *
     * @param string $fileName
     * @return string
     * @since 101.0.4
     */
    public static function getDispersionPath($fileName)
    {
        $char = 0;
        $dispertionPath = '';
        while ($char < 2 && $char < strlen($fileName)) {
            if (empty($dispertionPath)) {
                $dispertionPath = '/' . ('.' == $fileName[$char] ? '_' : $fileName[$char]);
            } else {
                $dispertionPath = self::_addDirSeparator(
                    $dispertionPath
                ) . ('.' == $fileName[$char] ? '_' : $fileName[$char]);
            }
            $char++;
        }
        return $dispertionPath;
    }
	
    /**
     * Add directory separator
     *
     * @param string $dir
     * @return string
     */
    protected static function _addDirSeparator($dir)
    {
        if (substr($dir, -1) != '/') {
            $dir .= '/';
        }
        return $dir;
    }

Thank you @wagnert for your efforts