devlog 01

I have some good and bad news for you...

The good one is that I made some progress since the last time we talked, search function is 90% complete, and the works are already readable!

The bad one... is that everything I did might be for nothing.

First, the good news!

It's functional! It's kinda good even. I'm finally able to see detailed search results and read complete works!

Here is the kindle browsing the main page of the project. The flask server is running off my main computer.

The search works! It shows all the basic info on the works: tags, warnings, categories, relationships, and more! And you can actually click on them and read!

(Also, check out this work, it's really great!)

Well, about that bad news...

The website is slow, like really slow.

I tested the project on my 10th gen kindle (which recieved a update that made the browser even slower without me noticing), and the website itself runs instantly. Of course it does, it's running locally in my network and the main page is just a little over 600 bytes (not kilobytes, just bytes). The problem is, as soon as I try to search for anything, the server takes its sweet time loading the results.

In fact, I just counted and it takes 20 seconds to show you the results.

When you click on a work it's fine -- if the story is short it takes less than a second to load, but the 20 seconds for a search is just insane and might as well invalidate the whole project.

I mean, one of the big reasons that I have to create aOld3 is to browse AO3 quicker on my kindle, so I can get to reading faster, but I honestly think this 20 seconds gets close to the time it takes for me to enter the true AO3 website, wait for it to load, search, and start reading.

Why is that the case?

I have no idea.

It could be my internet connection, but I don't think so. Don't get me wrong, my internet is terrible, but AO3's website search loads as usual, taking like 1 second max. So it can't be it.

Maybe the api i'm using is limiting how many requests I can make? Since the start of the project I was limiting the requests with the api tool ao3.utils.limit_requests(), but after seeing the crazy amounts of loading I turned it off with ao3.utils.limit_requests(False). And it made ZERO diference.

So I don't know.

This situation is even more annoying because since the project is still so early into development, I'm doing a lot of searches to test everything, and every single time I need to test anything I lose 20 seconds doing nothing, just waiting for the search results to pop up.

I really want to fix this, but I also don't want to anger AO3's protections. If I push too much (somehow, I don't even know how I could do it), the servers might tag me as someone trying to do a DDoS attack or trying to scrape the website for AI and then block me. Especially since a while ago, cloudflare is taking care of AO3. And they definitely would block me, just like they did in the early testing before I started using the api, throwing error 525 at me.

Well, it's getting late, and my back is killing me. tomorrow I will try to fix this somehow.

For today, I will just publish this devlog and call it quits, I need to read some AO3