you didnt understand the post and no only syao is trolling.petopeto said:
Yes, the dupe check does take a little intelligence to use.
You would be the third person here to tell me to stfu...
I think you set a fire under his ass.MDGeist said:
you didnt understand the post and no only syao is trolling.
You would be the third person here to tell me to stfu...
No need to get angry about it. Some people have concerns about the new naming schema.syaoran-kun said:
wth..just use an md5 hasher if you want hash filenames instead of bitching about the new change (that imho is better, cuz md5 filenames don't tell you a shit about the image itself)..what's the problem..too HARD for you?
Fyi i'm not trolling, i'm just trying to reply instead of facepalming to death.
My only issue is that all my nearly 10,000 images I have downloaded from here, no longer have have the same file names as the site. Because of this I am going to end up with a ton of duplicates when I accidentally re-download something which I didn't know I already had. Could someone at please make a renaming batch file to rename all the files with the md5 file names to their new file names?
While I wouldn't have had much of an issue if the file names were like this from the start, as it is now this is going to make for a huge waste of time and bandwidth. People are going to either have to rename everything, re-download everything, and/or waste space with thousands of identical images with the different names.
While I wouldn't have had much of an issue if the file names were like this from the start, as it is now this is going to make for a huge waste of time and bandwidth. People are going to either have to rename everything, re-download everything, and/or waste space with thousands of identical images with the different names.
You gotta be selective, Cyerbeing. ^^
Well about a good 6000+ of them are from the moe.imouto.org siterip from awhile back but for the rest I was selective and only downloaded what I like. There are about 4,188 images so far that I manually selected and downloaded by hand. Lets just say I have have a lot of free time and I enjoy archiving images.
For that reason (because I am selective and don't just download everything in order) this becomes an issue because I now have no way of knowing what I have already downloaded and what I haven't.
For that reason (because I am selective and don't just download everything in order) this becomes an issue because I now have no way of knowing what I have already downloaded and what I haven't.
Was trying to be humorous, then I get smack down. Thanks. I sure understand what these image mongers are like now. Then you should have saved the site rip and the selective image into the same folder. Just backtrack when the site rip was ripped and then you have a dateline.Cyberbeing said:
Well about a good 6000+ of them are from the moe.imouto.org siterip from awhile back but for the rest I was selective and only downloaded what I like. There are about 4,188 images so far that I manually selected and downloaded by hand. Lets just say I have have a lot of free time and I enjoy archiving images.
For that reason (because I am selective and don't just download everything in order) this becomes an issue because I now have no way of knowing what I have already downloaded and what I haven't.
W/e I have no reason to argue with some in incased with anger like you.
*A ha, now I finally understand that idioms " People are are afriad to change" when it's meant to ease them.
ps: Start fresh, have fun now :D
Knowing that the site rip was done in October 2007 doesn't really help me because I randomly download things sorted by tags as well as just page by page...
I'm not angry that the change was made because this is more of a personal problem then anything else. I'm more just venting frustration because it's unlikely it will go back to the old way (considering the new way is much better) and because I didn't see this coming I'm now in a real mess going forward. I'll be able to work around the problem it's just going to create many hours/days/weeks of unneeded work for me that I wasn't expecting nor am I looking forward to. Not very fun XD
If someone will either make a renaming batch file or torrent an updated site rip at some point I would be very thankful.
I'm not angry that the change was made because this is more of a personal problem then anything else. I'm more just venting frustration because it's unlikely it will go back to the old way (considering the new way is much better) and because I didn't see this coming I'm now in a real mess going forward. I'll be able to work around the problem it's just going to create many hours/days/weeks of unneeded work for me that I wasn't expecting nor am I looking forward to. Not very fun XD
If someone will either make a renaming batch file or torrent an updated site rip at some point I would be very thankful.
A Python script is fine too?Cyberbeing said:
If someone will either make a renaming batch file or torrent an updated site rip at some point I would be very thankful.
http://orphelin.no-ip.org/md5totags.py
When you run the script it asks for a folder. It searches that folder for images with md5 names, does an md5 search on imouto for those images, finds the new filenames, puts the old and new names in rename.txt and opens it for you so you can check if everything's fine and fix things that aren't before renaming.
It should only mess up if there are Japanese characters in a tag. They look fine in rename.txt but not in the actual filename after renaming (戦女神ＺＥＲＯ becomes 戦女神ＺＥＲＯ), you'll have to manually fix that.
Let me know if there are any problems.
I should dump the tags+post ids so you can look up locally instead of having to constantly hit the db.
Please don't make a separate request for every file; it'll load the server unnecessarily. Batch them into a single request: "md5:a,b,c,d". I don't think there's a hardcoded limit to the number of MD5s (but you may need to specify a higher limit if it's more than 16).
You'd be better off using the XML or JSON API than parsing HTML.
You'd be better off using the XML or JSON API than parsing HTML.
Thank you Biohazix, I tested your script on a few images and it works well. This will save me a lot of time.
As it seems the python script isn't very kind to the server, I'll hold off on using it for awhile. Is anybody able to modify a kinder script as petopeto suggested or make a new script to use the csv dump?
That got me thinking. If someone is willing to make a script for it, admin2, could you maybe once a month generate a dump with all the images that had their tags changed which contains both the old tags and updated tags in it (if that is even possible)? An automated script to do renaming for tag changes could be very useful and would negate another problem the new system causes.
As it seems the python script isn't very kind to the server, I'll hold off on using it for awhile. Is anybody able to modify a kinder script as petopeto suggested or make a new script to use the csv dump?
That got me thinking. If someone is willing to make a script for it, admin2, could you maybe once a month generate a dump with all the images that had their tags changed which contains both the old tags and updated tags in it (if that is even possible)? An automated script to do renaming for tag changes could be very useful and would negate another problem the new system causes.
I'll modify it to use the csv. Anything else I should change? Maybe include subdirectories, I don't know if that's useful to you.Cyberbeing said:
Is anybody able to modify a kinder script as petopeto suggested or make a new script to use the csv dump?
Edit: Done. http://orphelin.no-ip.org/md5totags.py
Put it in the same folder as dump.csv.
Thanks for that dump admin2. It's very fast now, takes about 9 seconds to rename 350 images for me.
A monthly dump would be nice. I could modify the script to update the filenames. (No need for old tags or only updated tags, a full dump like this one will do fine.)Cyberbeing said:
That got me thinking. If someone is willing to make a script for it, admin2, could you maybe once a month generate a dump with all the images that had their tags changed which contains both the old tags and updated tags in it (if that is even possible)? An automated script to do renaming for tag changes could be very useful and would negate another problem the new system causes.
I can setup a monthly export of the tags, its not too hard to have it export as csv.
that would be cool
admin2, would it be possible to export the csv as UTF-8 so the japanese and other unicode characters in the tags are preserved?
Also while running the new python script I ran into a couple problems but nothing I couldn't workaround.
The first is when the script runs into an error, it silently fails and closes. If it could be modified to ignore errors and continue renaming and then afterwards create a log file of the errors, that would be a preferable behavior.
Things that make it throw an error and close are:
Invaild characters in the tags \ / " | < > ? *
Tags which contain over the maximum supported characters for a file name.
Also while running the new python script I ran into a couple problems but nothing I couldn't workaround.
The first is when the script runs into an error, it silently fails and closes. If it could be modified to ignore errors and continue renaming and then afterwards create a log file of the errors, that would be a preferable behavior.
Things that make it throw an error and close are:
Invaild characters in the tags \ / " | < > ? *
Tags which contain over the maximum supported characters for a file name.
Looks like I should've tested more. Good idea, I'll make it log errors. Any other suggestions?Cyberbeing said:
Things that make it throw an error and close are:
Invaild characters in the tags \ / " | < > ? *
Tags which contain over the maximum supported characters for a file name.
Well I'm not sure how the site does it, but if you could just mimic its behavior for replacing invalid characters and shortening overly long tags in file names automatically that would probably be the best solution.
I already finished renaming everything with that csv based python script you posted so I don't even need it anymore (I just had to run it multiple times for every error it ran into that I forgot to fix) but it would be helpful for anybody else that needs to run it.
I also noticed another bug which I'm not sure of the cause.
For example the following which is outputted to the rename.txt:
02bcd4a0fd0ce3c30888a0267e2e838e.jpg | moe 4979 fate/stay_night koyama_hirokazu matou_sakura see_through stick_poster.jpg
The / is invalid so I changed it to the followed and saved the rename.txt:
02bcd4a0fd0ce3c30888a0267e2e838e.jpg | moe 4979 fate_stay_night koyama_hirokazu matou_sakura see_through stick_poster.jpg
If I then press enter, it crashes (I'm not sure if it even throws an error) even though everything should be fine. Now if I make the change directly in the csv and then run the python script it works correctly. Any idea why that happens?
A separate python script to just run off the rename.txt without parsing the csv might also be nice.
I already finished renaming everything with that csv based python script you posted so I don't even need it anymore (I just had to run it multiple times for every error it ran into that I forgot to fix) but it would be helpful for anybody else that needs to run it.
I also noticed another bug which I'm not sure of the cause.
For example the following which is outputted to the rename.txt:
02bcd4a0fd0ce3c30888a0267e2e838e.jpg | moe 4979 fate/stay_night koyama_hirokazu matou_sakura see_through stick_poster.jpg
The / is invalid so I changed it to the followed and saved the rename.txt:
02bcd4a0fd0ce3c30888a0267e2e838e.jpg | moe 4979 fate_stay_night koyama_hirokazu matou_sakura see_through stick_poster.jpg
If I then press enter, it crashes (I'm not sure if it even throws an error) even though everything should be fine. Now if I make the change directly in the csv and then run the python script it works correctly. Any idea why that happens?
A separate python script to just run off the rename.txt without parsing the csv might also be nice.
The only reserved characters that show up in tags are ", / and :. The script will automatically replace those just like imouto does. (Why is a : replaced with a space instead of an underscore? Makes it look like separate tags.)Cyberbeing said:
Well I'm not sure how the site does it, but if you could just mimic its behavior for replacing invalid characters and shortening overly long tags in file names automatically that would probably be the best solution.
If there's still a reserved character, or if the name is too long, it will say so in log.txt.
You'll have to manually shorten filenames if needed, I don't know how imouto does it.
Editing rename.txt doesn't do anything anymore because that shouldn't be necessary anymore, and to prevent errors (like that bug, no idea why that happened).
Sorry to bump to topic but does anyone still have that rename script?
The original link is dead QQ
The original link is dead QQ
dump.csv could need an update, too...
http://pastebin.com/f43413970Cerb69 said:
Sorry to bump to topic but does anyone still have that rename script?
The original link is dead QQ
If there would be regular updates of dump.csv I could modify it to update files with new tags, if anyone wants that.
I would still like that, but it seems admin2 has forgotten to setup the monthly csv dump of tags like he said he was going to do.
admin2, if you're still able to do monthly dumps of tags, I would be grateful.
admin2, if you're still able to do monthly dumps of tags, I would be grateful.
petopeto