Comments (5)
It appears that you have passed in a parameter scale
, please remove it and retry the generation.
from segmoe.
Sorry, just added that because I was hoping I could debug the issue (originally ran without). That was neither the problem, nor did it fix the problem.
I strongly suspect the problem is I am running a different version of diffusers or transformers than you intended
diffusers version: 0.26.1
transformers version: 4.37.2
bin C:\Users\Nagol\AppData\Local\Programs\Python\Python310\lib\site-packages\bitsandbytes\libbitsandbytes_cuda121.dll
0%
0/25 [00:00<?, ?it/s]
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
Cell In[2], line 7
5 prompt = "cosmic canvas, orange city background, painting of a chubby cat"
6 negative_prompt = "nsfw, bad quality, worse quality"
----> 7 img = pipeline(
8 prompt=prompt,
9 negative_prompt=negative_prompt,
10 height=1024,
11 width=1024,
12 num_inference_steps=25,
13 guidance_scale=7.5,
14 ).images[0]
15 img
File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\segmoe\main.py:866, in SegMoEPipeline.__call__(self, *args, **kwds)
860 """
861 Inference the SegMoEPipeline.
862
863 Calls diffusers.DiffusionPipeline forward with the keyword arguments. See https://github.com/segmind/segmoe#usage for detailed usage.
864 """
865 kwds["scale"]=7.5
--> 866 return self.pipe(*args, **kwds)
File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\utils\_contextlib.py:115, in context_decorator.<locals>.decorate_context(*args, **kwargs)
112 @functools.wraps(func)
113 def decorate_context(*args, **kwargs):
114 with ctx_factory():
--> 115 return func(*args, **kwargs)
File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\diffusers\pipelines\stable_diffusion\pipeline_stable_diffusion.py:1027, in StableDiffusionPipeline.__call__(self, prompt, height, width, num_inference_steps, timesteps, guidance_scale, negative_prompt, num_images_per_prompt, eta, generator, latents, prompt_embeds, negative_prompt_embeds, ip_adapter_image, output_type, return_dict, cross_attention_kwargs, guidance_rescale, clip_skip, callback_on_step_end, callback_on_step_end_tensor_inputs, **kwargs)
1024 latent_model_input = self.scheduler.scale_model_input(latent_model_input, t)
1026 # predict the noise residual
-> 1027 noise_pred = self.unet(
1028 latent_model_input,
1029 t,
1030 encoder_hidden_states=prompt_embeds,
1031 timestep_cond=timestep_cond,
1032 cross_attention_kwargs=self.cross_attention_kwargs,
1033 added_cond_kwargs=added_cond_kwargs,
1034 return_dict=False,
1035 )[0]
1037 # perform guidance
1038 if self.do_classifier_free_guidance:
File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py:1518, in Module._wrapped_call_impl(self, *args, **kwargs)
1516 return self._compiled_call_impl(*args, **kwargs) # type: ignore[misc]
1517 else:
-> 1518 return self._call_impl(*args, **kwargs)
File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py:1527, in Module._call_impl(self, *args, **kwargs)
1522 # If we don't have any hooks, we want to skip the rest of the logic in
1523 # this function, and just call forward.
1524 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
1525 or _global_backward_pre_hooks or _global_backward_hooks
1526 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1527 return forward_call(*args, **kwargs)
1529 try:
1530 result = None
File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\diffusers\models\unets\unet_2d_condition.py:1121, in UNet2DConditionModel.forward(self, sample, timestep, encoder_hidden_states, class_labels, timestep_cond, attention_mask, cross_attention_kwargs, added_cond_kwargs, down_block_additional_residuals, mid_block_additional_residual, down_intrablock_additional_residuals, encoder_attention_mask, return_dict)
1118 if is_adapter and len(down_intrablock_additional_residuals) > 0:
1119 additional_residuals["additional_residuals"] = down_intrablock_additional_residuals.pop(0)
-> 1121 sample, res_samples = downsample_block(
1122 hidden_states=sample,
1123 temb=emb,
1124 encoder_hidden_states=encoder_hidden_states,
1125 attention_mask=attention_mask,
1126 cross_attention_kwargs=cross_attention_kwargs,
1127 encoder_attention_mask=encoder_attention_mask,
1128 **additional_residuals,
1129 )
1130 else:
1131 sample, res_samples = downsample_block(hidden_states=sample, temb=emb, scale=lora_scale)
File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py:1518, in Module._wrapped_call_impl(self, *args, **kwargs)
1516 return self._compiled_call_impl(*args, **kwargs) # type: ignore[misc]
1517 else:
-> 1518 return self._call_impl(*args, **kwargs)
File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py:1527, in Module._call_impl(self, *args, **kwargs)
1522 # If we don't have any hooks, we want to skip the rest of the logic in
1523 # this function, and just call forward.
1524 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
1525 or _global_backward_pre_hooks or _global_backward_hooks
1526 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1527 return forward_call(*args, **kwargs)
1529 try:
1530 result = None
File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\diffusers\models\unets\unet_2d_blocks.py:1199, in CrossAttnDownBlock2D.forward(self, hidden_states, temb, encoder_hidden_states, attention_mask, cross_attention_kwargs, encoder_attention_mask, additional_residuals)
1197 else:
1198 hidden_states = resnet(hidden_states, temb, scale=lora_scale)
-> 1199 hidden_states = attn(
1200 hidden_states,
1201 encoder_hidden_states=encoder_hidden_states,
1202 cross_attention_kwargs=cross_attention_kwargs,
1203 attention_mask=attention_mask,
1204 encoder_attention_mask=encoder_attention_mask,
1205 return_dict=False,
1206 )[0]
1208 # apply additional residuals to the output of the last pair of resnet and attention blocks
1209 if i == len(blocks) - 1 and additional_residuals is not None:
File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py:1518, in Module._wrapped_call_impl(self, *args, **kwargs)
1516 return self._compiled_call_impl(*args, **kwargs) # type: ignore[misc]
1517 else:
-> 1518 return self._call_impl(*args, **kwargs)
File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py:1527, in Module._call_impl(self, *args, **kwargs)
1522 # If we don't have any hooks, we want to skip the rest of the logic in
1523 # this function, and just call forward.
1524 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
1525 or _global_backward_pre_hooks or _global_backward_hooks
1526 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1527 return forward_call(*args, **kwargs)
1529 try:
1530 result = None
File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\diffusers\models\transformers\transformer_2d.py:391, in Transformer2DModel.forward(self, hidden_states, encoder_hidden_states, timestep, added_cond_kwargs, class_labels, cross_attention_kwargs, attention_mask, encoder_attention_mask, return_dict)
379 hidden_states = torch.utils.checkpoint.checkpoint(
380 create_custom_forward(block),
381 hidden_states,
(...)
388 **ckpt_kwargs,
389 )
390 else:
--> 391 hidden_states = block(
392 hidden_states,
393 attention_mask=attention_mask,
394 encoder_hidden_states=encoder_hidden_states,
395 encoder_attention_mask=encoder_attention_mask,
396 timestep=timestep,
397 cross_attention_kwargs=cross_attention_kwargs,
398 class_labels=class_labels,
399 )
401 # 3. Output
402 if self.is_input_continuous:
File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py:1518, in Module._wrapped_call_impl(self, *args, **kwargs)
1516 return self._compiled_call_impl(*args, **kwargs) # type: ignore[misc]
1517 else:
-> 1518 return self._call_impl(*args, **kwargs)
File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py:1527, in Module._call_impl(self, *args, **kwargs)
1522 # If we don't have any hooks, we want to skip the rest of the logic in
1523 # this function, and just call forward.
1524 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
1525 or _global_backward_pre_hooks or _global_backward_hooks
1526 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1527 return forward_call(*args, **kwargs)
1529 try:
1530 result = None
File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\diffusers\models\attention.py:329, in BasicTransformerBlock.forward(self, hidden_states, attention_mask, encoder_hidden_states, encoder_attention_mask, timestep, cross_attention_kwargs, class_labels, added_cond_kwargs)
326 cross_attention_kwargs = cross_attention_kwargs.copy() if cross_attention_kwargs is not None else {}
327 gligen_kwargs = cross_attention_kwargs.pop("gligen", None)
--> 329 attn_output = self.attn1(
330 norm_hidden_states,
331 encoder_hidden_states=encoder_hidden_states if self.only_cross_attention else None,
332 attention_mask=attention_mask,
333 **cross_attention_kwargs,
334 )
335 if self.norm_type == "ada_norm_zero":
336 attn_output = gate_msa.unsqueeze(1) * attn_output
File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py:1518, in Module._wrapped_call_impl(self, *args, **kwargs)
1516 return self._compiled_call_impl(*args, **kwargs) # type: ignore[misc]
1517 else:
-> 1518 return self._call_impl(*args, **kwargs)
File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py:1527, in Module._call_impl(self, *args, **kwargs)
1522 # If we don't have any hooks, we want to skip the rest of the logic in
1523 # this function, and just call forward.
1524 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
1525 or _global_backward_pre_hooks or _global_backward_hooks
1526 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1527 return forward_call(*args, **kwargs)
1529 try:
1530 result = None
File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\diffusers\models\attention_processor.py:512, in Attention.forward(self, hidden_states, encoder_hidden_states, attention_mask, **cross_attention_kwargs)
493 r"""
494 The forward method of the `Attention` class.
495
(...)
507 `torch.Tensor`: The output of the attention layer.
508 """
509 # The `Attention` class can call different attention processors / attention functions
510 # here we simply pass along all tensors to the selected processor class
511 # For standard processors that are defined here, `**cross_attention_kwargs` is empty
--> 512 return self.processor(
513 self,
514 hidden_states,
515 encoder_hidden_states=encoder_hidden_states,
516 attention_mask=attention_mask,
517 **cross_attention_kwargs,
518 )
File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\diffusers\models\attention_processor.py:1224, in AttnProcessor2_0.__call__(self, attn, hidden_states, encoder_hidden_states, attention_mask, temb, scale)
1221 hidden_states = attn.group_norm(hidden_states.transpose(1, 2)).transpose(1, 2)
1223 args = () if USE_PEFT_BACKEND else (scale,)
-> 1224 query = attn.to_q(hidden_states, *args)
1226 if encoder_hidden_states is None:
1227 encoder_hidden_states = hidden_states
File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py:1518, in Module._wrapped_call_impl(self, *args, **kwargs)
1516 return self._compiled_call_impl(*args, **kwargs) # type: ignore[misc]
1517 else:
-> 1518 return self._call_impl(*args, **kwargs)
File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py:1527, in Module._call_impl(self, *args, **kwargs)
1522 # If we don't have any hooks, we want to skip the rest of the logic in
1523 # this function, and just call forward.
1524 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
1525 or _global_backward_pre_hooks or _global_backward_hooks
1526 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1527 return forward_call(*args, **kwargs)
1529 try:
1530 result = None
TypeError: SparseMoeBlock.forward() missing 1 required positional argument: 'scale'
from segmoe.
Seems to be an issue with the MoE layer forwards., we will look into it. The package was developed on the 0.26.0.devo version of diffusers, if that helps in any way for the time being.
from segmoe.
thanks!
It appears if I just change this
Line 44 in 9c06d7c
to
def forward(self, hidden_states: torch.Tensor, scale=None) -> torch.Tensor:
it works.
I'm not sure why the scale parameter is getting passed, given it isn't used in the function.
from segmoe.
Should be fixed with PR #14, Thanks for reporting!
from segmoe.
Related Issues (20)
- Minor mistake in readme HOT 1
- Thank you! + model suggestion HOT 2
- Issue with Civitai downloads HOT 2
- Any benefit to implementing this with lycoris/lora instead of full models? HOT 2
- Support local safetensors file HOT 1
- Support Colab and Local Storage HOT 1
- TypeError: no_grad.__init__() on import HOT 3
- positive and negative keywords in .yaml files HOT 2
- Is torch 2.0 mandatory? HOT 2
- [feature] Support StableDiffusionImg2ImgPipeline HOT 1
- Does this work for Stable Cascade? HOT 1
- 77 token limit HOT 5
- Why using negative prompt hidden states as gate weight? HOT 2
- Could you explain the effect of Pos and Neg prompts of each experts? HOT 2
- How to choose the positive prompt and negative prompt? HOT 2
- How to finetune the segmoe and train lora HOT 1
- Got noise image sample
- MoE in the attn heads
- Can you support SD3? HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from segmoe.