Giter Club home page Giter Club logo

krita-ai-diffusion's Introduction

Generative AI for Krita

โœจFeatures | โญณ Download | ๐Ÿ› ๏ธInstallation | ๐ŸŽž๏ธ Video | ๐Ÿ–ผ๏ธScreenshots | ๐Ÿ“–Wiki | ๐Ÿ’ฌDiscussion

Generate images from within Krita with minimal fuss: Select an area, push a button, and new content that matches your image will be generated. Or expand your canvas and fill new areas with generated content that blends right in. Text prompts are optional. No tweaking required!

This plugin seeks to provide what "Generative Fill/Expand" do in Photoshop - and go beyond. Adjust strength to refine existing content (img2img) or generate images from scratch. Powerful customization is available for advanced users.

Local. Open source. Free.

Watch video demo

Features

Features are designed to fit an interactive workflow where AI generation is used as just another tool while painting. They are meant to synergize with traditional tools and the layer stack.

  • Inpaint: Use Krita's selection tools to mark an area and remove or replace existing content in the image. Simple text prompts can be used to steer generation.
  • Outpaint: Extend your canvas, select a blank area and automatically fill it with content that seamlessly blends into the existing image.
  • Generate: Create new images from scratch by decribing them with words or existing images. Supports SD1.5 and SDXL.
  • Refine: Use the strength slider to refine existing image content instead of replacing it entirely. This also works great for adding new things to an image by painting a (crude) approximation and refining at high strength!
  • Live Painting: Let AI interpret your canvas in real time for immediate feedback. Watch Video
  • Control: Guide image creation directly with sketches or line art. Use depth or normal maps from existing images or 3D scenes. Transfer character pose from snapshots. Control composition with segmentation maps.
  • Resolutions: Work efficiently at any resolution. The plugin will automatically use resolutions appropriate for the AI model, and scale them to fit your image region.
  • Upscaling: Upscale and enrich images to 4k, 8k and beyond without running out of memory.
  • Job Queue: Depending on hardware, image generation can take some time. The plugin allows you to queue and cancel jobs while working on your image.
  • History: Not every image will turn out a masterpiece. Preview results and browse previous generations and prompts at any time.
  • Strong Defaults: Versatile default style presets allow for a simple UI which covers many scenarios.
  • Customization: Create your own presets - select a Stable Diffusion checkpoint, add LoRA, tweak samplers and more.

Getting Started

The plugin comes with an integrated installer for the Stable Diffusion backend.

Requirements

  • Windows, Linux, MacOS
  • On Linux/Mac: Python + venv must be installed
    • recommended version: 3.11 or 3.10
    • usually available via package manager, eg. apt install python3-venv

Hardware support

To run locally a powerful graphics card with at least 6 GB VRAM is recommended. Otherwise generating images will take very long!

NVIDIA GPUsupported via CUDA
AMD GPUsupported via DirectML on Windows, ROCm on Linux (only custom server)
Apple M1/M2supported via MPS on macOS
CPUsupported, but very slow
Cloud GPUsupported, rent a GPU on an hourly basis, see below

Installation

  1. If you haven't yet, go and install Krita! Required version: 5.2.0 or newer
  2. Download the plugin. Unpack the archive into your pykrita folder.
    • Windows: Usually C:\Users\<user>\AppData\Roaming\krita\pykrita
    • Linux: Usually ~/.local/share/krita/pykrita
    • MacOS: Usually ~/Library/Application Support/krita/pykrita
    • Check Krita's official documentation if you have trouble locating it.
  3. Enable the plugin in Krita (Settings โ€ฃ Configure Krita โ€ฃ Python Plugins Manager) and restart.
  4. Create a new document or open an existing image.
  5. To show the plugin docker: Settings โ€ฃ Dockers โ€ฃ AI Image Generation.
  6. In the plugin docker, click "Configure" to start server installation. Requires 10+ GB free disk space.

Note

If you encounter problems please check the FAQ / list of common issues for solutions.

Optional: Custom ComfyUI Server

The plugin uses ComfyUI as backend. As an alternative to the automatic installation, you can install it manually or use an existing installation. If the server is already running locally before starting Krita, the plugin will automatically try to connect. Using a remote server is also possible this way.

Please check the list of required extensions and models to make sure your installation is compatible.

Optional: Object selection tools (Segmentation)

If you're looking for a way to easily select objects in the image, there is a separate plugin which adds AI segmentation tools.

GPU Cloud

You can also rent a GPU instead of running locally. In that case, step 6 is not needed. Instead use the plugin to connect to a remote server.

There is a step by step guide on how to setup cloud GPU on runpod.io or vast.ai.

Screenshots

Inpainting on a photo using a realistic model

Reworking and adding content to an AI generated image

Adding detail and iteratively refining small parts of the image

Using ControlNet to guide image generation with a crude scribble

Modifying the pose vector layer to control character stances (Click for video) Watch video demo

Upscaling to improve image quality and add details

Server installation

Style preset configuration

Contributing

Contributions are very welcome! Check the contributing guide to get started.

Technology

krita-ai-diffusion's People

Contributors

acly avatar danamir avatar daniel-richter avatar fezinaru avatar gergelyszaz avatar huchenlei avatar jellydreams avatar miabrahams avatar sejoung avatar vtvrv avatar yantoz avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.