Giter Club home page Giter Club logo

Comments (5)

Warlord-K avatar Warlord-K commented on August 28, 2024

It appears that you have passed in a parameter scale, please remove it and retry the generation.

from segmoe.

nagolinc avatar nagolinc commented on August 28, 2024

Sorry, just added that because I was hoping I could debug the issue (originally ran without). That was neither the problem, nor did it fix the problem.

I strongly suspect the problem is I am running a different version of diffusers or transformers than you intended

diffusers version: 0.26.1
transformers version: 4.37.2

bin C:\Users\Nagol\AppData\Local\Programs\Python\Python310\lib\site-packages\bitsandbytes\libbitsandbytes_cuda121.dll

0%
0/25 [00:00<?, ?it/s]

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[2], line 7
      5 prompt = "cosmic canvas, orange city background, painting of a chubby cat"
      6 negative_prompt = "nsfw, bad quality, worse quality"
----> 7 img = pipeline(
      8     prompt=prompt,
      9     negative_prompt=negative_prompt,
     10     height=1024,
     11     width=1024,
     12     num_inference_steps=25,
     13     guidance_scale=7.5,
     14 ).images[0]
     15 img

File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\segmoe\main.py:866, in SegMoEPipeline.__call__(self, *args, **kwds)
    860 """
    861 Inference the SegMoEPipeline.
    862 
    863 Calls diffusers.DiffusionPipeline forward with the keyword arguments. See https://github.com/segmind/segmoe#usage for detailed usage.
    864 """
    865 kwds["scale"]=7.5
--> 866 return self.pipe(*args, **kwds)

File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\utils\_contextlib.py:115, in context_decorator.<locals>.decorate_context(*args, **kwargs)
    112 @functools.wraps(func)
    113 def decorate_context(*args, **kwargs):
    114     with ctx_factory():
--> 115         return func(*args, **kwargs)

File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\diffusers\pipelines\stable_diffusion\pipeline_stable_diffusion.py:1027, in StableDiffusionPipeline.__call__(self, prompt, height, width, num_inference_steps, timesteps, guidance_scale, negative_prompt, num_images_per_prompt, eta, generator, latents, prompt_embeds, negative_prompt_embeds, ip_adapter_image, output_type, return_dict, cross_attention_kwargs, guidance_rescale, clip_skip, callback_on_step_end, callback_on_step_end_tensor_inputs, **kwargs)
   1024 latent_model_input = self.scheduler.scale_model_input(latent_model_input, t)
   1026 # predict the noise residual
-> 1027 noise_pred = self.unet(
   1028     latent_model_input,
   1029     t,
   1030     encoder_hidden_states=prompt_embeds,
   1031     timestep_cond=timestep_cond,
   1032     cross_attention_kwargs=self.cross_attention_kwargs,
   1033     added_cond_kwargs=added_cond_kwargs,
   1034     return_dict=False,
   1035 )[0]
   1037 # perform guidance
   1038 if self.do_classifier_free_guidance:

File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py:1518, in Module._wrapped_call_impl(self, *args, **kwargs)
   1516     return self._compiled_call_impl(*args, **kwargs)  # type: ignore[misc]
   1517 else:
-> 1518     return self._call_impl(*args, **kwargs)

File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py:1527, in Module._call_impl(self, *args, **kwargs)
   1522 # If we don't have any hooks, we want to skip the rest of the logic in
   1523 # this function, and just call forward.
   1524 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
   1525         or _global_backward_pre_hooks or _global_backward_hooks
   1526         or _global_forward_hooks or _global_forward_pre_hooks):
-> 1527     return forward_call(*args, **kwargs)
   1529 try:
   1530     result = None

File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\diffusers\models\unets\unet_2d_condition.py:1121, in UNet2DConditionModel.forward(self, sample, timestep, encoder_hidden_states, class_labels, timestep_cond, attention_mask, cross_attention_kwargs, added_cond_kwargs, down_block_additional_residuals, mid_block_additional_residual, down_intrablock_additional_residuals, encoder_attention_mask, return_dict)
   1118     if is_adapter and len(down_intrablock_additional_residuals) > 0:
   1119         additional_residuals["additional_residuals"] = down_intrablock_additional_residuals.pop(0)
-> 1121     sample, res_samples = downsample_block(
   1122         hidden_states=sample,
   1123         temb=emb,
   1124         encoder_hidden_states=encoder_hidden_states,
   1125         attention_mask=attention_mask,
   1126         cross_attention_kwargs=cross_attention_kwargs,
   1127         encoder_attention_mask=encoder_attention_mask,
   1128         **additional_residuals,
   1129     )
   1130 else:
   1131     sample, res_samples = downsample_block(hidden_states=sample, temb=emb, scale=lora_scale)

File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py:1518, in Module._wrapped_call_impl(self, *args, **kwargs)
   1516     return self._compiled_call_impl(*args, **kwargs)  # type: ignore[misc]
   1517 else:
-> 1518     return self._call_impl(*args, **kwargs)

File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py:1527, in Module._call_impl(self, *args, **kwargs)
   1522 # If we don't have any hooks, we want to skip the rest of the logic in
   1523 # this function, and just call forward.
   1524 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
   1525         or _global_backward_pre_hooks or _global_backward_hooks
   1526         or _global_forward_hooks or _global_forward_pre_hooks):
-> 1527     return forward_call(*args, **kwargs)
   1529 try:
   1530     result = None

File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\diffusers\models\unets\unet_2d_blocks.py:1199, in CrossAttnDownBlock2D.forward(self, hidden_states, temb, encoder_hidden_states, attention_mask, cross_attention_kwargs, encoder_attention_mask, additional_residuals)
   1197 else:
   1198     hidden_states = resnet(hidden_states, temb, scale=lora_scale)
-> 1199     hidden_states = attn(
   1200         hidden_states,
   1201         encoder_hidden_states=encoder_hidden_states,
   1202         cross_attention_kwargs=cross_attention_kwargs,
   1203         attention_mask=attention_mask,
   1204         encoder_attention_mask=encoder_attention_mask,
   1205         return_dict=False,
   1206     )[0]
   1208 # apply additional residuals to the output of the last pair of resnet and attention blocks
   1209 if i == len(blocks) - 1 and additional_residuals is not None:

File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py:1518, in Module._wrapped_call_impl(self, *args, **kwargs)
   1516     return self._compiled_call_impl(*args, **kwargs)  # type: ignore[misc]
   1517 else:
-> 1518     return self._call_impl(*args, **kwargs)

File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py:1527, in Module._call_impl(self, *args, **kwargs)
   1522 # If we don't have any hooks, we want to skip the rest of the logic in
   1523 # this function, and just call forward.
   1524 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
   1525         or _global_backward_pre_hooks or _global_backward_hooks
   1526         or _global_forward_hooks or _global_forward_pre_hooks):
-> 1527     return forward_call(*args, **kwargs)
   1529 try:
   1530     result = None

File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\diffusers\models\transformers\transformer_2d.py:391, in Transformer2DModel.forward(self, hidden_states, encoder_hidden_states, timestep, added_cond_kwargs, class_labels, cross_attention_kwargs, attention_mask, encoder_attention_mask, return_dict)
    379         hidden_states = torch.utils.checkpoint.checkpoint(
    380             create_custom_forward(block),
    381             hidden_states,
   (...)
    388             **ckpt_kwargs,
    389         )
    390     else:
--> 391         hidden_states = block(
    392             hidden_states,
    393             attention_mask=attention_mask,
    394             encoder_hidden_states=encoder_hidden_states,
    395             encoder_attention_mask=encoder_attention_mask,
    396             timestep=timestep,
    397             cross_attention_kwargs=cross_attention_kwargs,
    398             class_labels=class_labels,
    399         )
    401 # 3. Output
    402 if self.is_input_continuous:

File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py:1518, in Module._wrapped_call_impl(self, *args, **kwargs)
   1516     return self._compiled_call_impl(*args, **kwargs)  # type: ignore[misc]
   1517 else:
-> 1518     return self._call_impl(*args, **kwargs)

File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py:1527, in Module._call_impl(self, *args, **kwargs)
   1522 # If we don't have any hooks, we want to skip the rest of the logic in
   1523 # this function, and just call forward.
   1524 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
   1525         or _global_backward_pre_hooks or _global_backward_hooks
   1526         or _global_forward_hooks or _global_forward_pre_hooks):
-> 1527     return forward_call(*args, **kwargs)
   1529 try:
   1530     result = None

File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\diffusers\models\attention.py:329, in BasicTransformerBlock.forward(self, hidden_states, attention_mask, encoder_hidden_states, encoder_attention_mask, timestep, cross_attention_kwargs, class_labels, added_cond_kwargs)
    326 cross_attention_kwargs = cross_attention_kwargs.copy() if cross_attention_kwargs is not None else {}
    327 gligen_kwargs = cross_attention_kwargs.pop("gligen", None)
--> 329 attn_output = self.attn1(
    330     norm_hidden_states,
    331     encoder_hidden_states=encoder_hidden_states if self.only_cross_attention else None,
    332     attention_mask=attention_mask,
    333     **cross_attention_kwargs,
    334 )
    335 if self.norm_type == "ada_norm_zero":
    336     attn_output = gate_msa.unsqueeze(1) * attn_output

File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py:1518, in Module._wrapped_call_impl(self, *args, **kwargs)
   1516     return self._compiled_call_impl(*args, **kwargs)  # type: ignore[misc]
   1517 else:
-> 1518     return self._call_impl(*args, **kwargs)

File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py:1527, in Module._call_impl(self, *args, **kwargs)
   1522 # If we don't have any hooks, we want to skip the rest of the logic in
   1523 # this function, and just call forward.
   1524 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
   1525         or _global_backward_pre_hooks or _global_backward_hooks
   1526         or _global_forward_hooks or _global_forward_pre_hooks):
-> 1527     return forward_call(*args, **kwargs)
   1529 try:
   1530     result = None

File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\diffusers\models\attention_processor.py:512, in Attention.forward(self, hidden_states, encoder_hidden_states, attention_mask, **cross_attention_kwargs)
    493 r"""
    494 The forward method of the `Attention` class.
    495 
   (...)
    507     `torch.Tensor`: The output of the attention layer.
    508 """
    509 # The `Attention` class can call different attention processors / attention functions
    510 # here we simply pass along all tensors to the selected processor class
    511 # For standard processors that are defined here, `**cross_attention_kwargs` is empty
--> 512 return self.processor(
    513     self,
    514     hidden_states,
    515     encoder_hidden_states=encoder_hidden_states,
    516     attention_mask=attention_mask,
    517     **cross_attention_kwargs,
    518 )

File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\diffusers\models\attention_processor.py:1224, in AttnProcessor2_0.__call__(self, attn, hidden_states, encoder_hidden_states, attention_mask, temb, scale)
   1221     hidden_states = attn.group_norm(hidden_states.transpose(1, 2)).transpose(1, 2)
   1223 args = () if USE_PEFT_BACKEND else (scale,)
-> 1224 query = attn.to_q(hidden_states, *args)
   1226 if encoder_hidden_states is None:
   1227     encoder_hidden_states = hidden_states

File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py:1518, in Module._wrapped_call_impl(self, *args, **kwargs)
   1516     return self._compiled_call_impl(*args, **kwargs)  # type: ignore[misc]
   1517 else:
-> 1518     return self._call_impl(*args, **kwargs)

File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py:1527, in Module._call_impl(self, *args, **kwargs)
   1522 # If we don't have any hooks, we want to skip the rest of the logic in
   1523 # this function, and just call forward.
   1524 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
   1525         or _global_backward_pre_hooks or _global_backward_hooks
   1526         or _global_forward_hooks or _global_forward_pre_hooks):
-> 1527     return forward_call(*args, **kwargs)
   1529 try:
   1530     result = None

TypeError: SparseMoeBlock.forward() missing 1 required positional argument: 'scale'

from segmoe.

Gothos avatar Gothos commented on August 28, 2024

Seems to be an issue with the MoE layer forwards., we will look into it. The package was developed on the 0.26.0.devo version of diffusers, if that helps in any way for the time being.

from segmoe.

nagolinc avatar nagolinc commented on August 28, 2024

thanks!

It appears if I just change this

def forward(self, hidden_states: torch.Tensor, scale) -> torch.Tensor:

to

def forward(self, hidden_states: torch.Tensor, scale=None) -> torch.Tensor:

it works.

I'm not sure why the scale parameter is getting passed, given it isn't used in the function.

from segmoe.

Warlord-K avatar Warlord-K commented on August 28, 2024

Should be fixed with PR #14, Thanks for reporting!

from segmoe.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.