Pre-filtering on shadow areas
The noise level of paper_texture & screening is directly proportional to darkness of images. thus darken only filtering & greyscale mask with inversion helps to avoid overfiltering.
#1 is the result of above post
First, duplicate a layer, then remove CrCb noises by your favorite filter (#2). it doesn't show the effect obviously, but it's essential.
2nd, duplicate 2nd layer again then set layer mode darken (#3)
3rd, apply Neatimage strongly at 3rd layer.
if a scan have paper_texture, up low-freq overly with very low-freq. the preview (#4) looks overfiltered, but no problem because the result will become #5 due to darken mode.
After merged 2nd & 3rd layers, add a back ground grayscale mask with inversion. mechanism is as same as the above post, only difference is it's with inversion (#6)
This is the end of pre-filtering. but it still needs main filtering.
after main filtered image is #7.
even though, It's not still finished. I have to do manual masking, dust erasing, smearing halo artifacts, e.t.c..
The noise level of paper_texture & screening is directly proportional to darkness of images. thus darken only filtering & greyscale mask with inversion helps to avoid overfiltering.
#1 is the result of above post
First, duplicate a layer, then remove CrCb noises by your favorite filter (#2). it doesn't show the effect obviously, but it's essential.
2nd, duplicate 2nd layer again then set layer mode darken (#3)
3rd, apply Neatimage strongly at 3rd layer.
if a scan have paper_texture, up low-freq overly with very low-freq. the preview (#4) looks overfiltered, but no problem because the result will become #5 due to darken mode.
After merged 2nd & 3rd layers, add a back ground grayscale mask with inversion. mechanism is as same as the above post, only difference is it's with inversion (#6)
This is the end of pre-filtering. but it still needs main filtering.
after main filtered image is #7.
even though, It's not still finished. I have to do manual masking, dust erasing, smearing halo artifacts, e.t.c..
Oh sweet. I'll have to try this out on the Moeoh scans next week, and maybe try the Neko Works book again.
(I edited your post a little bit to fix some typos)
(I edited your post a little bit to fix some typos)
In addition, my main filtering is Greycstration + Neatimage + USM2.
I'm doing this process twice with different settings, and 2nd process has a manual mask.
I'm doing this process twice with different settings, and 2nd process has a manual mask.
Results are somewhat mixed, but this is one that turned out well. I did a pass of Greycstoration and NI before resizing, and running midzki's action set without adjusting settings on #2.
Pre-filtering for reducing ringing artifacts of Neatimage.
Neatimage has terrible ringing artifacts on darkest pixels around edges, but masked Greycstoration helps to reduce it if it applied before Neatimage
Apply Greycstoration set -p 0.2 -a 0.9 ＼(＞ヮ＜) on a dupe layer, then add a mask from grayscale of a background with inversion, then lower gamma at 0.1 (or 0.05 might be better, but PS require 2 pass leveling in this case)
it strongly smooth out only darkest pixels.
Just a info because of my laziness
Neatimage has terrible ringing artifacts on darkest pixels around edges, but masked Greycstoration helps to reduce it if it applied before Neatimage
Apply Greycstoration set -p 0.2 -a 0.9 ＼(＞ヮ＜) on a dupe layer, then add a mask from grayscale of a background with inversion, then lower gamma at 0.1 (or 0.05 might be better, but PS require 2 pass leveling in this case)
it strongly smooth out only darkest pixels.
Just a info because of my laziness
It may be stupid question, but how to add a background layer mask? I don't know how to call out the window on #6
Under the (I)mage menu -->apply imge.castle said:
It may be stupid question, but how to add a background layer mask? I don't know how to call out the window on #6
Yeah I wasted much time on finding it too..
Got it, thanksfireattack said:
Under the (I)mage menu -->apply image.
Yeah I wasted much time on finding it too..
In case anyone missed it, this can be done more flexibly now with the USM2 plugin (forum #4685). Lower Amount+ to reduce brightening. Setting it to 0 is the same as Darken.midzki said:
if you got annoyed by side effects of USM, just use "Darken only" layer mode.
Your results are amazing midzki, but there's a few ambiguities that need to be clarified.
For example, to which layer should the filter be applied, from which layer should a duplicate be created, the sequence of layers, and from which layer should the mask be created, and to which layer should it be applied. Although someone who understands the principle lying beneath your processing is not likely to mess up, it would still help if the processing is made clear.
For example, consider there is a white scratch on a black background, if you create a mask from the unfiltered background, the white scratch will be retained because it's white and is masked from filtering. Creating from the filtered layer avoids this problem.
Also, about the relationship between noise level and brightness, I think it's not always the case with screening. An example is this image:
The darkest areas contain much less noise than the highlight areas in this image. Your assumption is quite reasonable because black ink can easily create a large contrast with other color inks and white paper, introducing high luma channel noise level. But if an area is dark enough, the noise level begins to decline with darkness level on the contrary. I think grayscale masking is a powerful method for preserving bright areas from overfiltering, but users should be aware that this masking method also has some deviation from real noise levels and should limit masking strength and look out for overfiltering in dark areas.
For example, to which layer should the filter be applied, from which layer should a duplicate be created, the sequence of layers, and from which layer should the mask be created, and to which layer should it be applied. Although someone who understands the principle lying beneath your processing is not likely to mess up, it would still help if the processing is made clear.
For example, consider there is a white scratch on a black background, if you create a mask from the unfiltered background, the white scratch will be retained because it's white and is masked from filtering. Creating from the filtered layer avoids this problem.
Also, about the relationship between noise level and brightness, I think it's not always the case with screening. An example is this image:
The darkest areas contain much less noise than the highlight areas in this image. Your assumption is quite reasonable because black ink can easily create a large contrast with other color inks and white paper, introducing high luma channel noise level. But if an area is dark enough, the noise level begins to decline with darkness level on the contrary. I think grayscale masking is a powerful method for preserving bright areas from overfiltering, but users should be aware that this masking method also has some deviation from real noise levels and should limit masking strength and look out for overfiltering in dark areas.
I rewrote the post a bit to clarify layer numbers.
it might not matter which I use. but in these kind of case, I'm prefer using noise level.
The problem of overfiltering on darks were maily caused by normal filtering.
Therefore, I'm using a opposite mask to avoid filtering on darks as filtering a whole image. because darks already haven't any noise after removed screening with that method.
for example,
apply one weak filter set => create a new layer with a mask WITHOUT inversion , then up gamma at 6~9 => apply another strong filter set at the new layer
In theory, using an unfiltered layer as a mask defines noise level, using a filtered layer as a mask defines noise reduction amount. but I couldn't find obvious difference between both ways.kiowa said:
For example, consider there is a white scratch on a black background, if you create a mask from the unfiltered background, the white scratch will be retained because it's white and is masked from filtering. Creating from the filtered layer avoids this problem.
it might not matter which I use. but in these kind of case, I'm prefer using noise level.
While I'm using only removing CrCb and Darken only mode, darks can avoid overfiltering because the ways hasn't white back.The darkest areas contain much less noise than the highlight areas in this image. Your assumption is quite reasonable because black ink can easily create a large contrast with other color inks and white paper, introducing high luma channel noise level. But if an area is dark enough, the noise level begins to decline with darkness level on the contrary. I think grayscale masking is a powerful method for preserving bright areas from overfiltering, but users should be aware that this masking method also has some deviation from real noise levels and should limit masking strength and look out for overfiltering in dark areas.
The problem of overfiltering on darks were maily caused by normal filtering.
Therefore, I'm using a opposite mask to avoid filtering on darks as filtering a whole image. because darks already haven't any noise after removed screening with that method.
for example,
apply one weak filter set => create a new layer with a mask WITHOUT inversion , then up gamma at 6~9 => apply another strong filter set at the new layer
if I use same filtering as normal paper for textured paper, ugly texture remains. so I thought I had a hand brush tool which can filtering only painted area.
then I set an action to imitate a filtering brush tool.
the way is simple.
dupe layer => select base layer then filtering => select top layer then reveal mask
all things I have to do other is tablet painting (￣▽￣)ノ
then I set an action to imitate a filtering brush tool.
the way is simple.
dupe layer => select base layer then filtering => select top layer then reveal mask
all things I have to do other is tablet painting (￣▽￣)ノ
denoise filters are based on blur methods, thus couldn't work well on RGB range 0~1, or 254~255. because the RGB range of the result images are closed in 1~254.
16bit filtering helps the result a bit, but still not good.
therefore, you may always need to clip out range 0~1 and 254~255 using levels.
16bit filtering helps the result a bit, but still not good.
therefore, you may always need to clip out range 0~1 and 254~255 using levels.
I just recently started to learn how to fix up scans, and I wanted to thank everyone for all of the information they share about it, especially Midzki! I'm also very grateful to petopeto for his Photoshop version of Greycstoration. Thank you!!
You are talking about quantization noise here, you can also consider adding a very weak noise (1~2 levels) to shift this quantization noise to a higher frequency range like they do with 16bit audio. Or just leave out a little high frequency noise when filtering.midzki said:
denoise filters are based on blur methods, thus couldn't work well on RGB range 0~1, or 254~255. because the RGB range of the result images are closed in 1~254.
16bit filtering helps the result a bit, but still not good.
therefore, you may always need to clip out range 0~1 and 254~255 using levels.
Adding noise is dithering, and you'd need to add 0.5 for it to work like you're thinking (adding 1 or more is just adding actual noise). I doubt that'll actually help here, but there's no raw in the above image for me to compare, so I'm not sure.
FM screening are dotted randomly to avoid moire, but it caused random hue shift on darks instead.
even if I used NI strongly, this hue shift arn't gone.
also some color mismatching problem between pages are probably FM screening's problem.
I think FM screening requires desaturate on darks.
I'm trying some ideas now.
for me, AM 200+lpi is the best screening more than FM screening now.
even if I used NI strongly, this hue shift arn't gone.
also some color mismatching problem between pages are probably FM screening's problem.
I think FM screening requires desaturate on darks.
I'm trying some ideas now.
for me, AM 200+lpi is the best screening more than FM screening now.
kiowa said:
You are talking about quantization noise here, you can also consider adding a very weak noise (1~2 levels) to shift this quantization noise to a higher frequency range like they do with 16bit audio. Or just leave out a little high frequency noise when filtering.
Wavelet based denoising (Neatimage, Noiseninja, Noiseware) can dither pixels. I'm using them after other denoizing in this reason.petopeto said:
Adding noise is dithering, and you'd need to add 0.5 for it to work like you're thinking (adding 1 or more is just adding actual noise). I doubt that'll actually help here, but there's no raw in the above image for me to compare, so I'm not sure.
Running Neatimage with 16bit mode helps the ugly result a bit.
However, black should be complete black, white should be complete white I think ideally.
1~2 is the mean strength applied to the whole image, and not every pixel get that much, so I guess I'll be fine. (With my current monitor I can't tell between 0 and 1 so I have no proof though -_-||| )petopeto said:
Adding noise is dithering, and you'd need to add 0.5 for it to work like you're thinking (adding 1 or more is just adding actual noise). I doubt that'll actually help here, but there's no raw in the above image for me to compare, so I'm not sure.
Maybe filtering at 16bit and try to dither the image when coming back to 8bit also helps, I don't know if photoshop does this well or just "nearest neighbour" 16bit levels to 8bit levels...
I confirmed conveting 16bit => 8bit on PS is dithering pixels. it'll become a bit noizy.
filtering with color range selection
I replaced my process of gray scale mask into color range selection.
There are no obvious difference in the result, but filtering is faster than masking.
the inline image shows a process of filtering with a selection from white
also added desaturate darks, and redden yellowish red processes. desaturate is applied color replacement simply, redden is applied color range selection.
I replaced my process of gray scale mask into color range selection.
There are no obvious difference in the result, but filtering is faster than masking.
the inline image shows a process of filtering with a selection from white
also added desaturate darks, and redden yellowish red processes. desaturate is applied color replacement simply, redden is applied color range selection.
:/
color range couldn't cover at middle range sufficiently.
though using it at reduce bleed through process is good, but failed at descreening process.
grayscale mask is better.
color range couldn't cover at middle range sufficiently.
though using it at reduce bleed through process is good, but failed at descreening process.
grayscale mask is better.
I can't believe why I didn't use an edge mask on Neatimage.
It can help from modifying edges by very-low-freq filtering.
find edges => surfaceblur => levels => gaussblur at the mask may work well.
if paper type is different, color range is different not just brightness but RGB balance, too.
my scanner is adjusted #1 type paper, but when I scan #2 type paper, they got strange hue even though actual paper has same colors in my eyes >_>
this result tells you why IT8 target failed, and why scans need naked eyes to adjust colors.
my scanner is adjusted #1 type paper, but when I scan #2 type paper, they got strange hue even though actual paper has same colors in my eyes >_>
this result tells you why IT8 target failed, and why scans need naked eyes to adjust colors.
Sure. first, your scanner detector's response is not quite the same as your eye's; second, the emission spectrum of your room light is quite different from your scanner's. So it's quite normal that real work results stray from target calibration results...midzki said:
if paper type is different, color range is different not just brightness but RGB balance, too.
my scanner is adjusted #1 type paper, but when I scan #2 type paper, they got strange hue even though actual paper has same colors in my eyes >_>
this result tells you why IT8 target failed, and why scans need naked eyes to adjust colors.
Making a target with EXACTLY the same paper and ink used by the book you are going scan maybe will make things better, but i don't think it practical...
Wikipedia describes this as "illuminant metameric failure". Images that look the same under some lighting (eg. halogen) may look different under other lighting (eg. the LCD light inside your scanner).
http://www.normankoren.com/color_management_3.html talks about one cause of this, which I suspect is the issue here:
http://www.normankoren.com/color_management_3.html talks about one cause of this, which I suspect is the issue here:
http://qualityinprint.blogspot.com/2009/03/issues-of-optical-brightening-agents-in.html talks about another: OBAs, which make a print look bluer under a light source containing UV. This is common in newspapers, apparently, and can be detected with a blacklight.spoiler
for your information, a IT8 target I got is kind of a laminated shirt. maybe it's imitating film like smooth papers.. never texturing doujin prints >_>
after I made many scans with this ways, and found color range selection is always worse than grayscale mask. it's can't erase noises well. guess because the range of calculation of denoizing filters is too small.midzki said:
:/
color range couldn't cover at middle range sufficiently.
though using it at reduce bleed through process is good, but failed at descreening process.
grayscale mask is better.
I use color range a lot when masking textured layers. For those, I usually want to filter dark areas, but not dark lines, to avoid smearing. I use color selection to pick the particular color ranges I need.
syaoran-kun