Thinking things through out-loud here, mostly, as I think at the moment
adevyish is the only other person with a real stake in this project (though if you are intrepid Kobo ereader owner who would like to easily browse and read fic on your device, you may too be interested!). There's basically two main challenges in this project - the first is that AO3 is designed for web, and only web, with the 'download this fic' button being the only way to access fic data in a different format, and the other is that Kobo firmware is basically a very minimal Linux OS, with no prebuilt web engine binary.
For issue two, the two solutions are 'write something to parse HTML and render it to an image' or 'use an HTML renderer someone else has written'. If I absolutely HAD to, I could do the first bit in Python - I've used the image-rendering packages in Python before, and I've even dealt with low-level interfaces for eink frame buffers. But, frankly, that's a LOT of work. Like we take reflowing text for granted, and it's only when you have to specify every linebreak yourself that you realize the number of calculations that go into it. Which leads us to 'use someone else's renderer', and for that, there are really three options: KoReader, which is written in C and Lua; Plato, which is written in Rust; and the Webkit engine that shipped with Qt4, which is C++. I actually tried the last option first, because I have a bit of experience with Qt from undergrad, but it turns out crosscompiling it is a huuuuuge pain in the ass and despite many hours attempting it I still haven't been able to succesfully crosscompile even sample code. Which leaves KoReader and Plato, and as I was going to be about equally lost in the code (as these are both single programs that happen to be OSS, rather than a general-purpose framework like Qt, and thus have basically no code documentation), I opted to go with Plato because I think going forward in my life, Rust will be handier, and Plato's GUI is nicer/more polished.
So - we've got
Plato for handling the HTML -> framebuffer rendering, and for all the GUI components, which leaves me to actually write the solution to problem one, fetching data from AO3 and rendering it into a format that is ereader-friendly, which basically involves fetching pages from AO3, parsing and scraping them for particular bits of content, and then feeding that cleaned content into Plato's HTML rendering engine. And then providing GUI elements that translate on the backend to specific requests to AO3, like posting a comment, leaving kudos, bookmarking a fic, etc. I really wish there were an API, because scraping and trying to mimic web traffic this way is a lot more brittle, but lol that's never gonna happen.
I went back and forth a bit in my head about whether I should be writing this as like, an app within Plato, or as a more complete fork, and I'm leaning towards fork, because it won't be possible for stuff from this to exist in the same views as local documents (as I explicitly do not want to save these requests as local documents - the data will only ever exist in memory, like it does in a browser), so I'm going to have to write my own 'Home' interface anyway. I'm kind of thinking of this in terms of views that need to be written, and what needs to be done for each one.
Work view- The minimum viable product, to use business-speak terms, and the bit I've actually started on
- Still struggling to decide if I want to fetch the whole work at once, or individual chapters. Drawbacks to whole-work: will have to modify the document tree to add relative anchor links to chapter titles, no easy access to per-chapter comments. Drawbacks to single-chapter: have to rewrite Plato's chapter functionality to handle remote locations rather than relative ones, have to figure out a nice way to move to the previous/next chapter in the reader when they're not part of the same document.
- I'm trying to figure out where to put access to AO3-specific functionality - the work metadata, kudos, comments, etc etc. I'd like to include it as buttons on the overlay controls, but there is already
quite a lot crammed in there already. I definitely don't want to show the work metadata on every chapter the way the web interface does, because on a limited screen size that would get obnoxious so very fast to page past.
- Maybe display controls should be moved into the dropdown? As you are substantially less likely to change them per-work than you are with other docs where the base formatting may vary.
List view- Basically any view of a list of works - a tag view, search results, an author's works, etc.
- How the fuck am I going to deal with wall-of-tags/variable lengths of summaries in such a limited space, I don't know
- I don't think I can fully proxy the advanced search page, because the auto-complete relies on XHR requests and I think the lag for making the requests and redrawing the UI is going to be too big. The sort-and-filter sidebar view is doable though because that data is populated on page load.
- Actions and metadata will vary based on the view, and go either at the top or bottom of the screens - space is somewhat less of an issue here because there'll be fewer display controls.
- Have to figure out how to handle pagination smoothly, because the AO3 results page size and the Kobo screen view page size will not be remotely the same, and will need to be tracked seperately in the backend.
Home view- The starting point for the app. Should handle login/logout functionality, and have paths to account view stuff, and browse.
- Going to replicate the favorite tag view, but ideally I'd like to add logic to be able to favorite ANY list view. Particularly filtered search views, so I don't have to keep resetting the filters to remove results in languages I cannot read. And AO3 refuses to add this functionality themselves, so.
There's probably also some assorted smaller views (like fandom browse), but I figure I'll get to those when I need them. When will this all get done? Who knows! But the idea won't leave me alone and reading fic on my phone in bed is not great for my eyes, my shoulders, or my sleep cycle, so I have incentive. And getting to the point of 'load HTML from memory, not from disk' was a major breakthrough so I have motivation to work on this.