I'm sure most of us have seen what generative fill in Photoshop can do, it's mighty impressive! However, the problem is you need to pay for a subscription and there's not a whole lot of control you have over the output. You just hit the button and hope you get a good result. It sure would be nice if there was a solution to both of these problems. It turns out, there is!

While I was searching for a hack to use generative fill for free, I stumbled across a plugin for an image editing program called Krita that uses Stable Diffusion to replicate generative fill in PS. I've looked into using SD to replicate generative fill before and the results have always been pretty crap. So imagine my surprise when this plugin not only works just as well as generative fill, but is actually better!

For starters, you can use a custom model, so you avoid the problem of generative fill getting confused and adding in photographic elements to illustrations. So for anime-style images, you get much better results right out of the gate by using a model trained on anime art. You've also got more control too. Instead of just pressing "generate" you can choose what type of generation you want to do. Expand? Fill? Remove? Refine? That last one is particularly useful, since generative fill can often get close, but you can't really fix any problems short of making tiny selections and fixing it bit-by-bit. The refine tool basically redoes what's already there, so you get a result that looks similar to the original, but maybe better. Oh, and this plugin also allows you to queue generations, so you don't need to babysit it if you want a ton of generations to choose from.

So what can you do with it? Well, I expanded yande.re/post/show/926298 to fit a 16:9 monitor by adding more content to the sides. Oh, and I also upscaled the image to 4K, yay again for A.I! I gotta say, the results look pretty good! Better than what I could do by hand!

https://imgur.com/a/xvtTKDI

Up-front, these results took a LONG time and a LOT of tweaking to get to this stage. You won't get results like this in one shot! A good trick I found is, rather than hoping you'll generate what you want if you keep hitting the generate button, what you do instead is give the A.I something to work off and use refine.

For example, the lanterns. Now, for the ones in the back, I can simply copy the ones from the original image and use fill and refine to integrate it into the image. But what about the one at the front? Ah, that's the trick! Since Stable Diffusion uses, well, diffusion, if you provide it with something that's roughly the shape and color of what you want, it'll use that to make the completed image. So all I needed to do was draw a box-shaped thing with yellow and brown shapes in the right places, and it looked for things of a similar shape and color in the image, resulting in the lantern. Though, the 3D depth was something it came up with, I was thinking it would be made of thin pieces of wood, but checking online, they're more square-shaped pieces of wood, with paper glued to the back. So I went with it.

Actually, speaking of the front lantern, the reason I put one there is because the A.I was constantly putting candles and lanterns in that spot when I was expanding the image. It was only later when I realized there was a glow on the original image that would indicate there would be a lantern off-screen. It's pretty impressive that the A.I picked up on that!

So yeah, that's my little experiment into A.I-powered image editing. I'd say this will be REALLY useful for fixing up images!

The main issue with the image is that it was sharpened a bit too much, getting rid of that kind of steamy blur which makes the floating leaves the artist did quite obvious, but other than that it looks fantastic! I also presume the plugin you're talking about is Acly/krita-ai-diffusion on github? I think there's a massive use case for AI when it comes to upscaling. Just need a good GPU for this kind of stuff which is going to become more of a problem.

between upscaling and generative fill a lot of images from the 5x4 (1280x1024 etc) era could be remastered on the fly.

still waiting for some ai thing i can easily run on my computer to stitch and clean new wallpapers.

Sonin said:
The main issue with the image is that it was sharpened a bit too much, getting rid of that kind of steamy blur which makes the floating leaves the artist did quite obvious, but other than that it looks fantastic!

Yeah, I sorta forced the A.I make everything sharp and not include steam. I had been having issues in other images where the output was always looking blurry and the A.I was including steam even in places where there shouldn't be steam. Going back and adding in some steam might have put the cherry on top!

Sonin said:
I also presume the plugin you're talking about is Acly/krita-ai-diffusion on github?

Yep, That's the one!

Abraxas said:
between upscaling and generative fill a lot of images from the 5x4 (1280x1024 etc) era could be remastered on the fly.

still waiting for some ai thing i can easily run on my computer to stitch and clean new wallpapers.

Assuming you have a decent enough GPU, you can run Waifu2x on your PC to upscale images with A.I. There's a lot of forks of this project, but I've found "Waifu2x-Extension-GUI" (https://github.com/AaronFeng753/Waifu2x-Extension-GUI) to be the best. It's got a ton more features and can even upscale video! But by far, it's best feature is you can select higher quality engines which eliminate the blurry, rainbowing on the line art that Waifu2x often produces.

@billyboy12 TBH that thing FUCKING SUUUCKS. Constant intrusive pop-ups and NOISE. And the output is SHIT.
It preserves noise artifacts and erases image details. Maybe the models aren't configured properly, but I was unable to get it to output any good quality images.

I use this simple thing: https://github.com/YukihoAA/waifu2x_snowshell
Less noise, more details. GRANTED, it is only for images, but holy fuck is the quality so much better, regardless of the model you choose.

snowshell
- Pros: No intrusive pop-ups, noise, better quality output.
- Cons: Only images, limits what scale you can use of some models.
Extension-GUI
- Pros: Can do video, can use any scale on any? model, more models to choose from.
- Cons: Pop-ups and NOISE. Shitty output.

Edit: Some more interesting findings:
CUGAN seems to really want straight, high-contrast lines. But that sometimes erases details or makes things up. It's also kinda noisy.
Caffe preserves a lot of details, but is blurrier/less sharp.
So... what I found is that Caffe 2x -> CUGAN 2x is almost universally better than just Caffe 4x or CUGAN 4x. (The order is important btw.)
P.S. ESRGAN just sucks (for anime anyway).

vonMooky said:
@billyboy12 TBH that thing FUCKING SUUUCKS. Constant intrusive pop-ups and NOISE. And the output is SHIT.
It preserves noise artifacts and erases image details. Maybe the models aren't configured properly, but I was unable to get it to output any good quality images.

I use this simple thing: https://github.com/YukihoAA/waifu2x_snowshell
Less noise, more details. GRANTED, it is only for images, but holy fuck is the quality so much better, regardless of the model you choose.

snowshell
- Pros: No intrusive pop-ups, noise, better quality output.
- Cons: Only images, limits what scale you can use of some models.
Extension-GUI
- Pros: Can do video, can use any scale on any? model, more models to choose from.
- Cons: Pop-ups and NOISE. Shitty output.

Edit: Some more interesting findings:
CUGAN seems to really want straight, high-contrast lines. But that sometimes erases details or makes things up. It's also kinda noisy.
Caffe preserves a lot of details, but is blurrier/less sharp.
So... what I found is that Caffe 2x -> CUGAN 2x is almost universally better than just Caffe 4x or CUGAN 4x. (The order is important btw.)
P.S. ESRGAN just sucks (for anime anyway).

Sweet merciful Jesus, are you ever right about how annoying Extension-GUI is! If snowshell can do CUGAN upscaling too, then hopefully it can act as a direct replacment, thanks for bringing it to my attention!

OK, I thought I'd try things out with a bigger challenge: converting a vertical image into a horizontal one! Specifically, this one:
https://yande.re/post/show/685541

This would require a LOT more generated image compared to just padding out the sides of an image that was already horizontal. Also, I was less strict with the A.I. In the previous image, I tried to only generate content that resembled what was already in the image, so as to better hide the fact that it had been padded out by A.I. This time, I thought I'd accept something if it looked good and made sense for what you would expect to be outside the borders of the original image.

Here's the results!
https://pixhost.to/show/971/471271493_yande-re-685541-digital_version-megane-meme50-naked-nipples-onsen-pussy-symmetri.png

I think it looks a bit nicer than simply making the rear fence infinitely long. I dunno, I think the A.I-generated bamboo fence even looks a bit better! The original is a bit too straight, real bamboo isn't so perfectly straight! I also like the heavy snow piling up all over the place, the original image does indicate there's heavy snow coverage.

The style mismatch still makes it look weird. The point about bamboo not being straight is valid, so maybe replacing the original image fence with the AI-style fence would look best. Or it could just be normal wood.

A bit off-topic - did you switch which upscaler you use? I wanted to ask opinions on which looks best. But since the same models seem to produce different output between GUIs, asking an opinion is only valid if you also use snowshell.

Anyway, if you do use snowshell which one looks best to you - caffe or cugan?

vonMooky said:
The style mismatch still makes it look weird. The point about bamboo not being straight is valid, so maybe replacing the original image fence with the AI-style fence would look best. Or it could just be normal wood.

A bit off-topic - did you switch which upscaler you use? I wanted to ask opinions on which looks best. But since the same models seem to produce different output between GUIs, asking an opinion is only valid if you also use snowshell.

Anyway, if you do use snowshell which one looks best to you - caffe or cugan?

It does look a bit weird, but I think it looks better than just having a really long fence by duplicating the original. Running refine on the original fence could have also worked to make it look less perfect and digital as well.

Yeah, I tried out snowshell! Their output looks the same to me, though snowshell is far less annoying, but at the same time, I'm not a fan of the simple "drop 'n' go" interface. Extension-GUI gives you more to tweak before you hit go. Plus, you have more resolution options than simply "2x" and "4x". Like, you can specify the horizontal/vertical resolution and it'll upscale the image to that size while maintaining aspect ratio. This is handy since I'm not upscaling the image any more than is necessary. I dunno, maybe there's technical reasons why upscaling in factors of 2 is better?

Definitely CUGAN! It does look a bit blurrier than CAFFE, but that can be reduced by lowering the noise reduction.

Scale factors depend on the used model.
CUGAN can do x2, x3 and x4. Snowshell is a bit buggy in that regard in not allowing x3.
ESRGAN can only do x4.
Caffe can do any scale.

Just go into the folders with the models - the file names are pretty self-explanatory.
Pretty sure extension-gui uses more traditional techniques to get to a specified resolution (like if you say 2.5, it could be doing x4, then shrinking back to 2.5).

Also, caffe looks much better to me, but that's because I'm pixel-peeping. But like I said before, caffe>cugan is probably best.

vonMooky said:
Scale factors depend on the used model.
CUGAN can do x2, x3 and x4. Snowshell is a bit buggy in that regard in not allowing x3.
ESRGAN can only do x4.
Caffe can do any scale.

Just go into the folders with the models - the file names are pretty self-explanatory.
Pretty sure extension-gui uses more traditional techniques to get to a specified resolution (like if you say 2.5, it could be doing x4, then shrinking back to 2.5).

Also, caffe looks much better to me, but that's because I'm pixel-peeping. But like I said before, caffe>cugan is probably best.

Yeah, it sounds like there's probably a good reason why most waifu2x apps only allow you to upscale by whole numbers. I think I'll stick to doing that then downscaling, I'm not seeing any artificing or anything doing this.

Oh, and on the topic of caffe vs cugan, I've just found that cugan is only really good for very low resolution source images, since it avoids the blurry and rainbowing on outlines. When used on high resolution images, it seems to make the lines a bit blurry, caffe seems to be better in this case.

A bit more experimenting! I was wondering if the A.I could add extra detail to rougher and more "unfinished" art to give it the more "finished" appearance it might have if the artist had more time to put into it (no offence to the artist!). I figured it should be possible, since the img2img mode makes the A.I use an existing image as its starting point and it expands on that, instead of using random noise to generate the image like in txt2img mode.

So I found what I think is a good test image over at https://yande.re/post/show/765004, (I mean the hair is mostly just a flat gradient and the shading on the skin is blocks of colors!) and did a quick onceover with the A.I to see what kinds results I'd get. I'd say it did the job nicely! Here's the results along with the original image for comparison:

https://pixhost.to/gallery/VAx4t

While it's a matter of opinion if this is "better" than the original art, it did indeed add the sorta extra detail that would have required more time from the artist. I'd also need to tinker with it a bit further to fix things like the changed facial expressions, the fence in the background being too short and the building in the background missing. I could also edit it further to make their eye color and hair color match the original image too.

vonMooky said:
Constant intrusive pop-ups and NOISE. And the output is SHIT.

If you just want to run waifu2x, there's https://unlimited.waifu2x.net/ now -- it's not as fast as locally, since it's in-browser, but that also means it's willing to accept larger images than the old server-based one.

The different models and denoising settings make a pretty big difference, so be sure to play around with them. I find it works pretty well, though it's definitely prone to a bit of haloing around sharp edges, and some settings will actually add a bit of noise to flat parts.

Okay, since I found out there's LORAs that are specifically intended for adding more detail to anime-style images, which is far more controllable than what I was doing, I've taken another look at using A.I to make simple, sketch artwork, look more finalized and polished artwork, the sorta thing that would have required FAR more time from the artist. Got a bit of a technique down for getting more detail, while not losing the bones of the original sketch. It's time-consuming, involving very gradual iterations and lots of touch-up, but it works pretty well!

I tried it out on this image:
https://yande.re/post/show/839077

Here's the results:
https://pixhost.to/show/316/496442053_yande-re-839077-sample-breasts-camonome-gym_uniform-nipples-no_bra-nopan-pussy-p.png

I think it turned out pretty good! When you look at thumbnails of the two images, they look quite similar! I even got the A.I to add little details like fingernails! And since the original image didn't have a background, I made one with A.I so it could be used as a wallpaper!

billyboy12 said:
Okay, since I found out there's LORAs that are specifically intended for adding more detail to anime-style images, which is far more controllable than what I was doing, I've taken another look at using A.I to make simple, sketch artwork, look more finalized and polished artwork, the sorta thing that would have required FAR more time from the artist. Got a bit of a technique down for getting more detail, while not losing the bones of the original sketch. It's time-consuming, involving very gradual iterations and lots of touch-up, but it works pretty well!

I tried it out on this image:
https://yande.re/post/show/839077

Here's the results:
https://pixhost.to/show/316/496442053_yande-re-839077-sample-breasts-camonome-gym_uniform-nipples-no_bra-nopan-pussy-p.png

I think it turned out pretty good! When you look at thumbnails of the two images, they look quite similar! I even got the A.I to add little details like fingernails! And since the original image didn't have a background, I made one with A.I so it could be used as a wallpaper!

Nice! What LORA did you use for that one (if you don't mind spilling the beans).

Sonin said:
Nice! What LORA did you use for that one (if you don't mind spilling the beans).

Beans spilt! It's Hyper detailer

https://civitai.com/models/196752/hyper-detailer