Whats the easiest/fastest way to get all xmls of a danboo?

limit=1000000 doesnt work, capped @ 1000 posts or so
pagewise kinda sucks (3000+ downloads just to refresh all tags)

ideal would be a range, like posts 0-1000, 1001-2000 and so on, with highest postcount to minimize pages to load... however, couldnt get it to work with tags+limit[xxx..yyy]

any ideas ?

api documentation isnt that much of a help

You can't get all of them at once. That's abusive to the server.

Scripts can read in bulk with limit=n&page=m. "limit=1000&page=2" will return posts 1001-2000.

If you're writing a script, the most efficient way to do it is to store the highest post ID you know about. Then, send the query "tags=id>N order:id holds:true", where N is your current highest post, to get the next batch of newer posts.

petopeto said:
Scripts can read in bulk with limit=n&page=m. "limit=1000&page=2" will return posts 1001-2000.

"tags=id>N order:id holds:true"

last one doesnt work
first one seems decent, although it sucks that page order isnt inverted.
"limit=1000&page=2" will NOT return posts 1001-2000 but posts
71095-72104

if the page ordering would start from 1 (first posts starting from id=3) to 3xxx , it´d be easier to see where one left imho, than checking by id

Although id based isnt that much of a problem either program-wise

Sure it works. http://moe.imouto.org/post?tags=id%3A%3E123+order%3Aid+holds%3Atrue

ah the : was missing
whats holds:true ?

after many tries i totally don´t trust danboorus api.
its as if sb progged it in 1 go, tested it with some files and it worked, but once it gets too big there are errors/missing files or simply other unexplainable shit :

like

"limit=1000&page=2" will return posts 1001-2000.

even this simple check returns shit with the main danboo :
http://danbooru.donmai.us/post/index.xml?limit=1000&page=388

<posts count="380223" offset="387000">
<post score="0" preview_width="150" tags="m

yeah sure, total posts 380k , when i start @387k and want the next 1000 i get... results! ;)

http://moe.imouto.org/post/index.xml?limit=1000&page=63 works fine though

However, i wouldnt be surprised if a run with my rename maker would show that posts are missing/or are duplicated ...

beta of my download prog works

way faster than rsync, too :) (no wonder, since its "multithreaded")

MDGeist said:
ah the : was missing
whats holds:true ?

Something to do with the post holding function?

Don't hammer the site with scripts.

The API works fine. There's a post page count cache which is generally glitchy, but that has nothing to do with the API.

does downloading with ~3-4mb/s from ranka count as hammering?

also seems like her update cycle is a bit slow, lotsa newer posts missing

admin2 wanted to post an automated sql/xml batch rar/zip, but guess he forgot or i forgot the dl url

3-4mb/s is *all* of ranka's sustainable bandwidth.

MDGeist, you are an utter idiot. Per request I can throw more insults at him. Personally I'd recommend banning him by IP.

petopeto said:
3-4mb/s is *all* of ranka's sustainable bandwidth.

thought it is a 100mbit ksufi

edit:
also, judging by all the dl links i got, out of 9413 posts only 2255 had ranka as downloadserver. ~24% ... out of the first 250 posts, none were hosted on ranka. too bad mrtg is broken atm, i doubt ranka pushes lots of traffic as she simply is not listed as download server as much as sheryl, but well what do i know. i dont want to download from sheryl, as european traffic should stay european!

MDGeist, you are an utter idiot. Per request I can throw more insults at him. Personally I'd recommend banning him by IP.

uhu... well dont panic, i wont release my new rename.txts , scripts for offline browsing and will never make a 150+ gig torrent again to help moes bw. happy ?

also great first forum post!

MDGeist said:
also great first forum post!

At least I found it funny.