I thought of using google map's interface idea of click+drag to navigate huge images with selective loading but I never really looked in to how to implement it.
HD space is cheap, bandwidth is cheap, in the near future we're moving to a gbit line over in europe instead of being hosted in states (expensive and makes me liable for copyright infringement)
I'm not sure what others think though. I use the resize option quite often to be honest so I can take a quick look.
The other problem I can see is that we'll be making more images of the same things so more chances of duping and such
HD space is cheap, bandwidth is cheap, in the near future we're moving to a gbit line over in europe instead of being hosted in states (expensive and makes me liable for copyright infringement)
I'm not sure what others think though. I use the resize option quite often to be honest so I can take a quick look.
The other problem I can see is that we'll be making more images of the same things so more chances of duping and such
I use resize almost every time, but it makes images look like crap in the browser, and doing it in advance (with a real resizer, not firefox's dumb one) would look much better.
Bandwidth is cheap, but it still takes 10-30 seconds for me to load most images (750k/sec line, but I usually get about 200k/sec here), and it still turns Firefox into a slow beast after doing it for a while. It ends up taking 400 megs of memory--even when no images are still loaded--and every tab click takes seconds to happen, and I'm not holding my breath hoping FF3 will fix this.
I guess the MD5 of the semi-thumb (still havn't thought of the right word) could be stored with the original post, so it could be checked on upload, too, though of course getting a good heuristic dupe checker is the real fix. (Has piespy released his image search stuff/tweaked imgseek? I wonder if anyone's working on integrating that for a dupe checker, I know there's been a lot of talk of it. I guess the GPL is why rq won't integrate it into trunk.)
I didn't understand "other problem"--what was the first problem?
If I can make it work with simple dupe checking, and nobody thinks of any more serious problems with the idea, would you consider it if I implemented it? (Seems like a fun day project, but it's not interesting to me if you won't use it.) It'd make the site a lot easier to use.
Bandwidth is cheap, but it still takes 10-30 seconds for me to load most images (750k/sec line, but I usually get about 200k/sec here), and it still turns Firefox into a slow beast after doing it for a while. It ends up taking 400 megs of memory--even when no images are still loaded--and every tab click takes seconds to happen, and I'm not holding my breath hoping FF3 will fix this.
I guess the MD5 of the semi-thumb (still havn't thought of the right word) could be stored with the original post, so it could be checked on upload, too, though of course getting a good heuristic dupe checker is the real fix. (Has piespy released his image search stuff/tweaked imgseek? I wonder if anyone's working on integrating that for a dupe checker, I know there's been a lot of talk of it. I guess the GPL is why rq won't integrate it into trunk.)
I didn't understand "other problem"--what was the first problem?
If I can make it work with simple dupe checking, and nobody thinks of any more serious problems with the idea, would you consider it if I implemented it? (Seems like a fun day project, but it's not interesting to me if you won't use it.) It'd make the site a lot easier to use.
Sure I'll implement it if its done, always open to improvements to the site (maybe I should run a svn)
The other problem that I was talking about is people reposting those resized scans elsewhere, you know contributing to more low res duplication, just a minor issue I thought of out of the blue for some reason and I don't know why I said other problem either heh.
Also firefox3 does handle large images a lot better and the resized results look better then firefox2
The other problem that I was talking about is people reposting those resized scans elsewhere, you know contributing to more low res duplication, just a minor issue I thought of out of the blue for some reason and I don't know why I said other problem either heh.
Also firefox3 does handle large images a lot better and the resized results look better then firefox2
IMO, reposting lower-res versions of scans is fine; most boards aren't looking for scans like these. If I uploaded a scan from here to another board, I'd probably resize it to a resolution typical of the board, and if a board has mostly 100k pictures and I upload a 3-meg one, I'd expect to get justly yelled at ...
An svn would be nice, better than coding against trunk and then having to merge, but probably no big deal.
An svn would be nice, better than coding against trunk and then having to merge, but probably no big deal.
Very true. Most boards won't accept such high res images, but that's what I love moeimouto for ^^ High res. Though I do have a tendency to post lower res iamges some times ^^, but high quality ^^
The obvious cautions apply: test in a sandbox first, back up your database, etc. I'd suggest glancing over the code to sanity check it, too; this is the first time I've used Ruby.
Installation:
make -C lib/danbooru_image_resizer
RAILS_ENV=xxx rake db:add_sample_columns
mkdir public/data/sample
Create sample images:
RAILS_ENV=xxx nice rake sample_images:create_missing
This will take a long time; until a sample exists for an image, it'll fall back on the full image. You can stop and restart this and it'll pick up where it left off. If you change CONFIG["sample_XXX"] settings, run sample_images:update_outdated.
Users can turn this off with "Show Image Samples" in settings. It defaults on.
A "High resolution" link is added to post/show. Clicking it switches the view to high-res; it also links to the file, so save-as, copy-location, etc. work.
All images are resized; PNG files can be fairly large, even at wallpaper resolution. It might be a waste for smaller images, but those are uncommon on moe, so I didn't want to complicate the code for it yet.
The resolution can be tweaked. The default height of 1000 already targets 1920x1200, so I wouldn't go any higher. I think 300k is a good target file size.
Only LocalFlat and LocalHierarchy stores are tested. RemoteHierarchy partially tested, don't have anything to try warehousing with.
Installation:
make -C lib/danbooru_image_resizer
RAILS_ENV=xxx rake db:add_sample_columns
mkdir public/data/sample
Create sample images:
RAILS_ENV=xxx nice rake sample_images:create_missing
This will take a long time; until a sample exists for an image, it'll fall back on the full image. You can stop and restart this and it'll pick up where it left off. If you change CONFIG["sample_XXX"] settings, run sample_images:update_outdated.
Users can turn this off with "Show Image Samples" in settings. It defaults on.
A "High resolution" link is added to post/show. Clicking it switches the view to high-res; it also links to the file, so save-as, copy-location, etc. work.
All images are resized; PNG files can be fairly large, even at wallpaper resolution. It might be a waste for smaller images, but those are uncommon on moe, so I didn't want to complicate the code for it yet.
The resolution can be tweaked. The default height of 1000 already targets 1920x1200, so I wouldn't go any higher. I think 300k is a good target file size.
Only LocalFlat and LocalHierarchy stores are tested. RemoteHierarchy partially tested, don't have anything to try warehousing with.
Its up and running but trying to generate the older images the rake fails, but new pictures coming in works fine, here's the trace:
[root@hatsune danbooru]# RAILS_ENV=production nice rake sample_images:create_missing --trace
(in /mnt/ayashii/moe/danbooru)1/13351 (post #16822) ...
- Invoke sample_images:create_missing (first_time)
- Invoke environment (first_time)
- Execute environment
- Execute sample_images:create_missing
rake aborted!
No such file or directory - /mnt/ayashii/moe/danbooru/config/../data/81456-sample.jpg or /mnt/ayashii/moe/danbooru/config/../public/data/sample/14/62/1462afc9854eca811c2aa221419b3bed.jpg
/usr/local/lib/ruby/1.8/fileutils.rb:505:in `rename'
/usr/local/lib/ruby/1.8/fileutils.rb:505:in `mv'
/usr/local/lib/ruby/1.8/fileutils.rb:1395:in `fu_each_src_dest'
/usr/local/lib/ruby/1.8/fileutils.rb:1411:in `fu_each_src_dest0'
/usr/local/lib/ruby/1.8/fileutils.rb:1393:in `fu_each_src_dest'
/usr/local/lib/ruby/1.8/fileutils.rb:494:in `mv'
/mnt/ayashii/moe/danbooru/app/models/post.rb:1017:in `regenerate_sample'
/mnt/ayashii/moe/danbooru/config/../lib/tasks/make_sample_images.rake:3:in `regen'
/mnt/ayashii/moe/danbooru/config/../lib/tasks/make_sample_images.rake:30
/mnt/ayashii/moe/danbooru/config/../lib/tasks/make_sample_images.rake:27:in `each'
/mnt/ayashii/moe/danbooru/config/../lib/tasks/make_sample_images.rake:27
/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:546:in `call'
/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:546:in `execute'
/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:541:in `each'
/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:541:in `execute'
/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:508:in `invoke_with_call_chain'
/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:501:in `synchronize'
/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:501:in `invoke_with_call_chain'
/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:494:in `invoke'
/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1931:in `invoke_task'
/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1909:in `top_level'
/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1909:in `each'
/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1909:in `top_level'
/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1948:in `standard_exception_handling'
/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1903:in `top_level'
/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1881:in `run'
/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1948:in `standard_exception_handling'
/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:1878:in `run'
/usr/local/lib/ruby/gems/1.8/gems/rake-0.8.1/bin/rake:31
/usr/local/bin/rake:19:in `load'
/usr/local/bin/rake:19
add
FileUtils.mkdir_p(File.dirname(sample_path), :mode => 0775)
before the FileUtils.mv in models/post.rb regenerate_sample
FileUtils.mkdir_p(File.dirname(sample_path), :mode => 0775)
before the FileUtils.mv in models/post.rb regenerate_sample
Thanks, looks like its working.
I was thinking instead of having high res link on the left bar we had something like this: http://imouto.org/resize.png
I was thinking instead of having high res link on the left bar we had something like this: http://imouto.org/resize.png
How about having the high res link say "High resolution (6000x4000, 12mb)", have text at the top that just says "this image has been resized, click High resolution to see the original [ ] click here to hide this message". That way, people are just informed about it at first per account (or cookie/system or whatever so it works for anon), maybe reenabling the message if they have samples turned off and then turn them back on. That'd keep from cluttering up the post page forever.
Can you put an svn up with the code? Making further patches from db trunk is a bit of a pain, much easier to make diffs from your code directly.
Can you put an svn up with the code? Making further patches from db trunk is a bit of a pain, much easier to make diffs from your code directly.
Sure I'll get that setup in a few.
Is it really a good idea for the smaller image have the exact same filename as the larger one? As the small image will have a different MD5 value. Shouldn't something like 'sample' be appended to the filename?
Yeah, some less perceptive people are definitely getting a little confused. I'll implement this tonight if you put svn up (let me know if the "tip" format I suggested sounds OK).
I expect that the image that appears after clicking the "high res" option isn't the final one and it's only a test because it's only a resize of the little image and very pixelated. I prefer the old way. I have enough seeing the thumbnail to know whether I what the image or not. I don't want to need to load two images to obtain the one I like.
What browser/version are you using? It'll show a pixellated one at first (as it'll scale the sample, at least in FF2), but it should load the full one.
You can go into your profile settings and turn it off if you want, but please let me know what your configuration is if the link isn't working for you.
You can go into your profile settings and turn it off if you want, but please let me know what your configuration is if the link isn't working for you.
hmm... a pretty decent idea, though a little odd for me...
i usually browse by small thumbnail, and then when i open the image, opera automatically resizes the display of the image to fit the width of my screen... does a decent job too, since only the image is resized, while the rest of the page is at normal size...
though, looking at one image i downloaded, the high res link is kinda hard to spot... it may need to be set out/defined more, to at least bring proper attention to it..
/me throws his two cents in teh pond...
i usually browse by small thumbnail, and then when i open the image, opera automatically resizes the display of the image to fit the width of my screen... does a decent job too, since only the image is resized, while the rest of the page is at normal size...
though, looking at one image i downloaded, the high res link is kinda hard to spot... it may need to be set out/defined more, to at least bring proper attention to it..
/me throws his two cents in teh pond...
Honestly, I prefer the original way of displaying large scan at once. At least it allows you to have a good view of the scan before you save it to your own HDD.
Though if it really saves bandwidth then I wouldnt complain :)
Though if it really saves bandwidth then I wouldnt complain :)
kab: Opera definitely does a better job than FF2 at resizing. I used Opera before, and I liked it (it's very fast), but I can't live without Firefox's add-ons now.
abao: You can see the full image in the browser before saving it, just click "high resolution". Let me know if that's not working for you.
This isn't really to save bandwidth. The biggest reason is to be able to see viewable (non-thumb) versions of the images interactively--it's so much nicer to open a picture and see it in a second or two (like other image boards), compared to 10-30 seconds. (It just might happen to save bandwidth too.)
admin2, patch coming: adding the hidable tip box to tell you you're seeing a sample, and a shortcut (if you're logged in) to turn off sample images, so people who don't like it can turn it off without hunting for it. The "original image" link will show the file resolution and size, and is a different color.
ed: admin2: is there a problem with saki? I'm getting 404s, and not just on samples (eg. http://saki.imouto.org/data/b4/8d/b48d49d1ff6a6e737e735b413f81a163.png).
abao: You can see the full image in the browser before saving it, just click "high resolution". Let me know if that's not working for you.
This isn't really to save bandwidth. The biggest reason is to be able to see viewable (non-thumb) versions of the images interactively--it's so much nicer to open a picture and see it in a second or two (like other image boards), compared to 10-30 seconds. (It just might happen to save bandwidth too.)
admin2, patch coming: adding the hidable tip box to tell you you're seeing a sample, and a shortcut (if you're logged in) to turn off sample images, so people who don't like it can turn it off without hunting for it. The "original image" link will show the file resolution and size, and is a different color.
ed: admin2: is there a problem with saki? I'm getting 404s, and not just on samples (eg. http://saki.imouto.org/data/b4/8d/b48d49d1ff6a6e737e735b413f81a163.png).
btw. what about rsync ?
i dont want to dl these "big previews" for backup...
i dont want to dl these "big previews" for backup...
I'm using firefox and until I have turned off the option I still had the pixelated resized image when I pressed the hi res option. Now with the option deselected the images load on hi res properlypetopeto said:
What browser/version are you using? It'll show a pixellated one at first (as it'll scale the sample, at least in FF2), but it should load the full one.
You can go into your profile settings and turn it off if you want, but please let me know what your configuration is if the link isn't working for you.
Yeah it's pixelated like mad for me to, was forced to make that change as well, as it's just a big pixelated pic othervise.
Added in a new patch from petopeto, now it shouldn't be as confusing
saki hasnt' been rsynced yet I was waiting for the samples to finish generating before getting that donepetopeto said:
ed: admin2: is there a problem with saki? I'm getting 404s, and not just on samples (eg. http://saki.imouto.org/data/b4/8d/b48d49d1ff6a6e737e735b413f81a163.png).
){kind=link}
Can you leave the image resize off as default? It is already confusing people.
I can still see http://moe.imouto.org/post/show/16823/miko in the post area after someone deleted it. Related issue?
I can still see http://moe.imouto.org/post/show/16823/miko in the post area after someone deleted it. Related issue?
I think I'm just being a bit slow today. You've started to use deletion placeholders...I can still see http://moe.imouto.org/post/show/16823/miko in the post area after someone deleted it. Related issue?
Its part of the migration to danbooru 1.13 it'll probably disappear again whenever the next changeset comes inRadioactive said:
I think I'm just being a bit slow today. You've started to use deletion placeholders...
Also I think there needs to be some sort of progress bar that shows up after you click on view original image not everyone will notice the inline loading.
Deletion placeholders are good. The post area starts to look a little weird after I've deleted a few dupes. Might be worth changing the image used for something else. Competition time perhaps?admin2 said:
Its part of the migration to danbooru 1.13 it'll probably disappear again whenever the next changeset comes in
Also I think there needs to be some sort of progress bar that shows up after you click on view original image not everyone will notice the inline loading.
Progress bar would be good.
It'd be good to leave it enabled by default, because nobody will ever even know it's there otherwise.
I want to make it just hide the image and load the original, not to show the sample scaled up in the interim like it does now. I havn't figured out how to do that yet, without creating a separate image tag for it (and that would break lots of javascript). Setting img.src to "" before setting it to the target image doesn't work. I'll look more into it, open to suggestions from any javascript people.
I don't like the placeholders, but apparently that's just a transitional thing in svn (see shuugo's post in forum #7 p25).
I want to make it just hide the image and load the original, not to show the sample scaled up in the interim like it does now. I havn't figured out how to do that yet, without creating a separate image tag for it (and that would break lots of javascript). Setting img.src to "" before setting it to the target image doesn't work. I'll look more into it, open to suggestions from any javascript people.
I don't like the placeholders, but apparently that's just a transitional thing in svn (see shuugo's post in forum #7 p25).
admin2: Sorry, I left a debug hack in that's making the "don't show this" link not work right. Just change "ahide_resized_notice" to "hide_resized_notice" in post/show.rhtml.
ok fixed that thanks.
petopeto
image previews
- image browsing would be easier; I like saving those 2000x6000 images, but for casual browsing (finding the images I want to have at 2000x6000), it's a bit painful, even at 200k/second. Huge images tend to slow down Firefox for me badly, too ...
- Images wouldn't have to be resized in the browser. I've heard FF3 is better at this (it wasn't when I tried it), but FF2 just has a nearest resize, so images look terrible when I resize them to fit. (People at lower resolutions would still have to, depending on the resolution cap set, but it'd be more reasonable to scroll around the image, too.)
- less bandwidth use if people aren't downloading full-resolution scans when they end up not caring about the image. I always save images I like locally, but I only actually want to do that for maybe one in 20 images I load, so that'd cut down my bandwidth use massively.
- piclens would be much more usable.
The current torrent is 11082 images at 29.3 gb: average 2.7m per image, so with 300k previews, this would be about a 10-15% disk space cost, plus time to resize the images on upload (and resizing existing images once).
(I don't know what to call these. Danbooru already calls thumbs "previews".)
I'm not sure how to interface it: not everyone would want this (but it wouldn't be much of a bandwidth savings if it's not on in post/show by default)--maybe a profile setting and a "full resolution" link from post/show. directlink would still go to the full image, of course.
If this sounds useful, I'm willing to give a shot at implementing it and sending a patch; the thumbs code in db seems fairly straightforward.