>>> [!note] Migrated issue <!-- Drupal.org comment --> <!-- Migrated from issue #3531212. --> Reported by: [marcus_johansson](https://www.drupal.org/user/385947) Related to !686 >>> <h3 id="summary-problem-motivation">Problem/Motivation</h3> <p>Create an image-to-image operation type that can work for all the DreamStudio type of operations, including:</p> <p>Upscale<br> Creative Upscale<br> Image to creative QR code<br> Inpaint<br> Outpaint<br> Remove Backgroun<br> Erase Object<br> Sketch to Image<br> Structure to Image<br> Recolor</p> <p>This means that we need a very flexible input and output for this. The only requirements will always be that the image is inputted and the image is outputted. In upscale models for instance, this might be the only required data.</p> <p>For more complex things like Inpaint a prompt or even a mask might be required.</p> <p>We create feature flags for models, similar to if tools calling works for instance. This means that the getModels of providers that support image to image has to be very flexible.</p> <p>This operation type will work very badly with default operation type kind of tasks, if this gets important, for instance for upscaling, we would add a pseude default operation type called image_to_image_upscale or something similar.</p> <p>Because its unclear during scoping what exactly will be needed this will be developed with AI API Explorer as part of the issue and with DreamStudio provider being used as the example for most use cases.</p> <h4 id="summary-steps-reproduce">Steps to reproduce</h4> <h3 id="summary-proposed-resolution">Proposed resolution</h3> <p>Create ImageToImage operation type directory<br> Create ImageToImageInput that require to take an abstract Image<br> Add Helper Traits for media<br> Create ImageToImageInput that returns an abstract Image<br> Create ImageToImageInterface with imageToImage method<br> Add a method called requiresImageToImageMask that sets a boolean<br> Add a trait for ImageToImage that sets this to false by default<br> Explore what the input actually needs, but make all those fields optional - the providers have to throw exception when something is missing and make sure that the model configs has required fields for mask, prompts etc.<br> Create an AI API Explorer plugin for this.<br> Add ModelCapabilityFlags as it progresses.</p> <h3 id="summary-remaining-tasks">Remaining tasks</h3> <h3 id="summary-ui-changes">User interface changes</h3> <h3 id="summary-api-changes">API changes</h3> <h3 id="summary-data-model-changes">Data model changes</h3> > Related issue: [Issue #3552212](https://www.drupal.org/node/3552212) > Related issue: [Issue #3456765](https://www.drupal.org/node/3456765)