I have the same problem. It feels like a bug to me.
Can you please check @checkmate and @edogawaconan?
Can you please check @checkmate and @edogawaconan?
Yes. In this moments is impossible view pictures after 1000 pages. I create a new account and the problem continue.
too many people accessing large page numbers causing overload on the server
Will it ever go back to normal? Still inable to access the pages.admin6 said:
too many people accessing large page numbers causing overload on the server
我猜站長有設下什麼保護措施吧
我已經看過好幾個地方號稱扒了整個站的圖片
這個站都已經是用愛發電了
禁得起這樣扒嗎?
記得以前曾經關站過不就是因為被扒到流量太高站長負擔不了流量費才關站
我已經看過好幾個地方號稱扒了整個站的圖片
這個站都已經是用愛發電了
禁得起這樣扒嗎?
記得以前曾經關站過不就是因為被扒到流量太高站長負擔不了流量費才關站
I believe the API design can be improved by using "Keyset Pagination" to reduce the load when viewing large page numbers:
https://medium.com/swlh/mongodb-pagination-fast-consistent-ece2a97070f3
Without access to the source code my educated guess is that the current implementation is using SKIP-LIMIT style of pagination by inspecting the URL:
https://yande.re/post?page=5
This roughly corresponds to the below SQL query:
SELECT *
FROM posts
ORDER BY id DESC
LIMIT 40
OFFSET 40 * 4
which will cause heavy DB load when viewing large page numbers as 40 * (page number) rows needs to be scanned and then skipped by the DB engine, just to return the next 40 posts.
Instead, we could do something like this:
SELECT *
FROM posts
WHERE id < 886857
ORDER BY id DESC
LIMIT 40
where post 886857 is the last post in page 4. This query can be executed very fast on the DB server without much load, because it is very likely the "id" column has a proper index to be utilized by the DB engine. In this case the URL is likely:
https://yande.re/post?after=886857
The downside is that it is not possible to show a row of page numbers at the bottom of the page to instantly jump to a particular page, since it is hard to know which starting id is for which page. It will instead just have a link of "Previous" and "Next" page.
@admin6 do you want to give this idea a try?
https://medium.com/swlh/mongodb-pagination-fast-consistent-ece2a97070f3
Without access to the source code my educated guess is that the current implementation is using SKIP-LIMIT style of pagination by inspecting the URL:
https://yande.re/post?page=5
This roughly corresponds to the below SQL query:
SELECT *
FROM posts
ORDER BY id DESC
LIMIT 40
OFFSET 40 * 4
which will cause heavy DB load when viewing large page numbers as 40 * (page number) rows needs to be scanned and then skipped by the DB engine, just to return the next 40 posts.
Instead, we could do something like this:
SELECT *
FROM posts
WHERE id < 886857
ORDER BY id DESC
LIMIT 40
where post 886857 is the last post in page 4. This query can be executed very fast on the DB server without much load, because it is very likely the "id" column has a proper index to be utilized by the DB engine. In this case the URL is likely:
https://yande.re/post?after=886857
The downside is that it is not possible to show a row of page numbers at the bottom of the page to instantly jump to a particular page, since it is hard to know which starting id is for which page. It will instead just have a link of "Previous" and "Next" page.
@admin6 do you want to give this idea a try?
it's already possible just there's no interface for it (search for id:..500 for example). Interface for it also has to consider actual order of the posts as it can be changed using order: search option.
also I have different priorities and this is low on my list.
also I have different priorities and this is low on my list.
Ah wasn't aware of "id:..500". Seems to be good enough for me!
now it shows can't go beyond pege more than 5000 to me.
i have browse it for more than one week
i have browse it for more than one week
this method have some bug i think .bao__zhe said:
Ah wasn't aware of "id:..500". Seems to be good enough for me!
i try input "id:..548535" and "id:..528535" , the result page is the same beginning from post #504965.
this means almost 50k post i have jump?
There is no bug. The order of post isn't by id because of howto:hold. You can use order:id_desc id:..528535 to force the ordering to be id desc.
thx.fireattack said:
There is no bug. The order of post isn't by id because of howto:hold. You can use order:id_desc id:..528535 to force the ordering to be id desc.
so if i dont input the order:id_desc , what does the order show at common pages?
Basically the same as id_desc but held images would be inserted at whenever they're released (unheld).Hercles said:
thx.
so if i dont input the order:id_desc , what does the order show at common pages?
If it helps, search can also be by date like so for example: date:2006-08-13 (that date also being the first date that there were any posts).
Is it listed by when the post was showed but not uploaded, just like id:..500?Jalawala said:
If it helps, search can also be by date like so for example: date:2006-08-13 (that date also being the first date that there were any posts).
yamichun
为什么不能浏览1000页之后的图片了？Why can't I browse pictures after 1000 pages?