Ao3 Reader 0.1.1

Wednesday, 2 October 2024 20:11
momijizukamori: Grey tabby cat with paws on keyboard and mouse. The text reads 'code cat is on the job', lolcats-style (CODE CAT)

Not a huge release, feature/bugfix-wise (though there's a few!), but lots of nice backend cleanup, and I'm trying to be more organized about documenting changes and whatnot, so we've got a fancy github release page this time! The zip file attached to that should be unzipped into an existing ao3-reader install folder - I will see about getting a new OCP package built and added to that, but I really want to automate the process a bit more.

Many thanks to [personal profile] shootthelightsout, [personal profile] qwertynerd97, and [github.com profile] diantox for contributing bug reports, QA, code and docs! I'm really excited that other people are interested in this project, which is very motivating :)

Ao3 Reader 0.1.0

Sunday, 12 May 2024 00:34
momijizukamori: Grey tabby cat with paws on keyboard and mouse. The text reads 'code cat is on the job', lolcats-style (CODE CAT)
Major features/bugfixes
- No longer crashes on rotation! Though custom views haven't had rotation code written for them mostly, so there will be weird rendering bugs in landscape mode.
- Support for devices through the Clara BW (so not the Clara Color or Libre Color, but everything else) - thanks upstream!
- The 'jump to page' feature of indexes (accesesed by tapping on the page count in the bottom right) now actually jumps you to the page instead of hanging
- The work details overlay (accessed by long-pressing on a work in the list) now actually shows all the tags and summary instead of cutting off partway into the second page.
- AO3 login! For now, you need to put your username and password in the .adds/ao3reader/Settings.toml file - in the future I'd like to have that info be enterable in the UI but we're not there yet.
- The above means 1) you can access archive-locked works, and 2) you can access your 'Marked For Later' list (currently at the end of the home page faves list) - that last one was a big thing I wanted, heh.
- A few bugfixes here and there - work listings with more than 1k works will now show the count properly, if a tag has been synned to another tag it'll handle the redirect instead of saying it's unwrangled, etc.
- Relevant only to people who want to do development on this, I figured out how to get the desktop emulator working (but it requires a bunch of system lib installs so not something I can just distribute for now, sigh)

Known Issues
- The summary view doesn't understand linebreaks, or HTML, so all the text is just one long paragraph. That's one of the next big goals
- Kudos are not working. Or working intermittantly? idk there's some weird session check stuff I was trying to debug today but I finally gave up
- About a million things remain unimplemented, lol

The installation instructions remain the same - the new 'one-click' package is here, and if you've already got an older version installed and just want to update the ao3reader install, the zip file is here (there's some library updates so it needs the whole folder, not just the binary this time).
momijizukamori: Grey tabby cat with paws on keyboard and mouse. The text reads 'code cat is on the job', lolcats-style (CODE CAT)
Because I know I have people here who are interested - I will hopefully have a new release this weekend, featuring fixes for the two most annoying longstanding bugs, along with support for newer devices and working Ao3 login.

(and if I don't post it please pester me about it, lol)
momijizukamori: Green icon with white text - 'I do believe in phosphorylation! I do!' with a string of DNA basepairs on the bottom (Default)
After a bunch of digging through compiler errors, I finally managed to pull in the upstream Plato updates and rebuild my app. No new features, this is just support for new devices.

If you do not have a Sage, Libra 2, or Clara 2E, I recommend installing the previous version. There's some minor bugs in the new one I haven't had time to work out yet, name:
- rotating the display makes the app crash
- works aren't getting proper styling for title/author/etc so all the text looks the same.

If you are willing to go forward with that, you can grab the 'one-click install' file.

The original install instructions work for both versions.
momijizukamori: Grey tabby cat with paws on keyboard and mouse. The text reads 'code cat is on the job', lolcats-style (CODE CAT)
Small update - full install as before, or if you have successfully installed already, you can download the binary alone and paste it into .adds/ao3reader/

Changelog
- Tweak to fontsize on tags, at [personal profile] adevyish's suggestion
- Whether the frontlight button takes screenshots or toggles the frontlight is now configurable via the 'screenshot_button' setting in Settings.toml - true makes it take screenshots, false (the default value) makes it toggle the frontlight
- Fix for works not rendering properly

The works bug turns out to be because while Plato opens .html files, the actual parser implementation is only for XHTML (which is what epub uses). As such, void tags (tags that can never have any content, and thus don't require a closing tag, like img or br) caused it to parse things badly, because it detected them as opening tags which were never closed. Thankfully the parser implementation was fairly straightforward and easy-to-read so it wasn't hard to add proper handling for void tags. I made an upstream PR with my changes, though if it gets rejected I won't be particularly fussed because I do realize it's kind of out of the main scope of Plato (which is epub and formats supported by MuPDF)


I spent a good chunk of my long weekend working on things that weren't this, so... maybe some other updates sometime soon?
momijizukamori: Green icon with white text - 'I do believe in phosphorylation! I do!' with a string of DNA basepairs on the bottom (Default)
This is now almost at the 'beta' stage, AKA the point where I start posting about it other places, though I have a small number of very troublesome bugs left, and I'd like to confirm that it doesn't turn into hot garbage on non-Glo devices before I send it out into the wild.

Changelog:
- Custom logo, scripts have now been renamed to 'Ao3Reader' instead of 'Plato'
- Tag navigation now works properly, both within indexes and works
- Second page of tags and summaries now available - sort of (see known issues)
- Won't crash if network unavailable or trying to load bad work index urls!
- Display settings for reader now save/load properly
- Saving index views to favorites via the star icon now works! You can also add them manually in Settings.toml - see the existing ones for an example
- Work index items now show chapter count
- You can now switch between long view (title, author, fandoms, and as many tags as will fit) and short view (title, author, fandoms) by setting work_display in Settings.toml to either "Long" or "Short"
- Page progress more accurately reflects position in the work (it was being calculated off the entire HTML size, which includes a pile of JS/layout bits that get thrown out)

Known Issues:
- Jump-to-page in the work index view does not work and will instead just hang weirdly
- When the about work overlay goes to multiple pages, it just... stops rendering partway down the second page? The data is there - if tags spill into that area you have invisible touch-targets for them - but they're not being drawn to the frame buffer for ?? reasons ??
- I've found at least one work that doesn't generate it's pages properly - I strongly suspect it's a combo of weird HTML and something in the underlying HTML engine (which is inherited untouched from Plato), but if you find works where the displayed text cuts off weirdly (particularly if it seems to be generating chapterlist entries for points past that), please drop me a link to the work so I have more data.

Install instructions remain as before (but look for 'Ao3 Reader' instead of 'Plato')
momijizukamori: Green icon with white text - 'I do believe in phosphorylation! I do!' with a string of DNA basepairs on the bottom (Default)
Same instructions as my previous post - the file that links to has been updated with the newer version.

Changelog:
- We now have live-rendering 'about work' info! At the moment you can only view the first page, and it's just title/author/fandoms/tags. I'm going to work on that some soon. To view, either use the 'About Work' toolbar item while reading a work, or hold your finger down on a row in the workindex to pop it up for that work.
- Navigation-by-tags works from the workindex view - though unfortunately there's no history yet.
- Works count in the bottom bar of workindex view
- Pls ignore the weird area at the top of the workindex view, it's SUPPOSED to contain the title of the index you're looking at (what Ao3 puts in the titlebar), but the render function is never being called for it, for ??reasons?? Likewise the little star doesn't do anything yet - eventually it'll let you add/remove an index view from your list of favorites.

Next steps for me are more info on the about overlay, and doing a whole bunch of code refactoring/cleanup - I'm having issues debugging the title bar bit because the code for the view that adds it is Too Messy.
momijizukamori: Grey tabby cat with paws on keyboard and mouse. The text reads 'code cat is on the job', lolcats-style (CODE CAT)
Because this is actually in a state where it's possible to share with other people for feedback! To install:
1) Download the kfmon/reader zip here.
2) Connect your Kobo via USB
3) Extract the zip directly into the base folder of the device (if, like me, you have an external SD card, you want the KOBOeReader device, that's the internal storage)
4) Add the following two lines to .kobo/Kobo/Kobo eReader.conf:

5) Eject device - it will appear to process imports and then run a short update (which is patching in the hooks for kfmon, which is the launcher util) and reboot.

Then to access, go into your Books and look for the 'book' (actually just an image file) named Plato and select it - that triggers the launcher script. To exit, use the menu in the top right corner -> quit

Caveats, etc
- You need to have the wifi on and be connected to a network before you try to launch the app - it'll crash immediately otherwise (I haven't put in error handling yet for when it can't fetch web pages)
- The physical frontlight button will not actually toggle the frontlight - instead it'll take a screenshot. You can toggle the frontlight from the menu bar though.
- You can only view one 'work index' (which is what I've been calling the listings you get from either searching or for looking at all works in a tag). It's set by default to the Sha Po Lang tag, but if you open .adds/plato/Settings.toml and go to the last line (base-path) you can set that to your URL of choice, either with the full domain name or relative to archiveofourown.org

This should be pretty safe - I've tested it a bunch, it's based on existing well-tested code, and it doesn't muck around much with system files. But I'm not responsible if something does go horribly wrong (though I will try to help fix), and you should back up your files regularly anyway, etc.

Midnight Edit: I FORGOT A STEP also the zip file was missing one of the necessary bits. If you got this before the edit and added like forty icons to your books list I am so so sorry, if you delete the folder and then add the config lines it'll resync properly
momijizukamori: Green icon with white text - 'I do believe in phosphorylation! I do!' with a string of DNA basepairs on the bottom (Default)
I figured out how to take screenshots within the app without having a menu in the way! First off, some aesthetic tweaks to the reader overlay:
images underneath )

Aaaaand I got the first step of the 'about work' overlay done, which is to take a list of tags and turn them into clickable elements, with wrapping. Which is actually kind of complicated when you have to manually set breakpoints and define locations absolutely!
More images! )

I'd really appreciate feedback on the element size - on one hand they feel REALLY big, but on the other, the touch resolution on eink screens are not great, and you can't pinch-zoom to get around it like you can on a phone etc. On the OTHER other hand, I'm on a Kobo Glo, which is like, the second-oldest series that's still supported, and the second-lowest resolution available out of these devices - it's 1024 x 758/4 3/4" x 3.5" for 212ppi, so it's possible that 'barely okay' on the Glo is plenty acceptable on all the other models.
momijizukamori: Grey tabby cat with paws on keyboard and mouse. The text reads 'code cat is on the job', lolcats-style (CODE CAT)

Giant gif under the cut )

Currently hard-coded to the Sha Po Lang tag (and, uh, overriding the exiting Home view), but that's not hard to fix once there's a mechanism for actually specifying the input. Unfortunately this does mean I can no longer put off dealing with the full work info overlay which is gonna be a pain, uggggggh. But it's the main thing standing in front of a workable beta I can give people.

(then I have to like, make actual designs about UI things. Gross.)
momijizukamori: Grey tabby cat with paws on keyboard and mouse. The text reads 'code cat is on the job', lolcats-style (CODE CAT)
I got live-built chapter-lists working! Also pagination of the chapter list. I also figured out how to turn videos on my phone into gifs for easier sharing.

cut to save you from loading a slightly choppy 5MB gif if you don't want to )

The thing I've been working on since getting that done is index views - they're kind of complicated because the page size is different between the e-reader and Ao3, and because you can't load everything at once, so you need to keep track of what's been loaded and what hasn't. Still kind of working that out, but I gave myself permission this weekend to do nothing, so I'm trying not to dwell on it and just release instead.
momijizukamori: (tits against the rte)
So it turns out that when you are used to reflowable HTML, having to write a GUI where you have to manually break lines yourself and absolutely position all elements is a huge pain in the ass. I begin to understand why everyone uses Electron these days. But I started to take a stab at work index entries, because I don't want to accept that I'm going to have to write a view for full work blurbs from scratch instead of using the HTML engine. Here's what we've got so far:


The black box is where an implementation of the required tags grid will go - it'll probably be smaller, but I need to do some testing to figure out what size it needs to be to be intelligable. It'll be a togglable option whether to show or hide. For the main body, I'm thinking of having two modes - one with up to two lines of title/author, and one line of fandoms, and one with up to two lines of title/author, up to two lines of fandoms, and remaining space filled with tags. Probably they will be on either side, size-wise, of the current prototype, as I realized that the proto is kind of an akward height.

Far right column is update date and word count at the moment, though I think I need to do something to make that clearer. Having those two items be configurable (replace with any of the other number stats - hits/kudoes/comments/bookmarks/chapters) is a definite possibility. Also gonna bump the tiny font up a little, I think.

Current plan is to have it open the work itself with a single tap, and to open up the overlay with the scrollable full blurb on long-press. But! For the people who were interested in this app, what would be your ideal items to show in the index view? Configuration beyond long/short and the required tags grid will probably come later but I am interested in what people want.

EDIT: I post this and then go to load a new version of the code onto my Glo only to discover that something has gone seriously wonky with it and while it will boot to the home screen, I can't interact with anything and the battery doesn't seem to be charging? I am going to leave it off and plugged in for a few hours and see if that helps at all, and then try a factory reset if it doesn't.

In other news...

Friday, 14 May 2021 22:55
momijizukamori: (tits against the rte)
I have had some time in the last week to work on my Ao3 Kobo app (see previously), and I have actually made a decent amount of progress! I finally figured out some of how to write UI components, so I've got it pulling data from a URL and loading it cleanly in a reader window, and updated a bunch of the UI to reflect AO3-specific functionality, including a working 'kudos' button. The next big steps will be figuring out how to do partial refreshing of UI elements, so I can add pagination/'scrolling' inside parts of the UI, and then building an interface for the chapters lists (which requires the UI bit first, so that when the list is too big to fit on the screen you can move to all the sections). Then making the work-blurb view actually readable (which also requires the UI bit).

Some photo-screenshots:


(also the kudos JS code is REAL bad, the main Rust http client package is kind of bad, and I have volunteered myself as tribute and sent a message via the feedback form to be like 'yo if you want a public API I will do it for you' because having to scrape over the HTML pages in Rust makes me want to set things on fire)
momijizukamori: Grey tabby cat with paws on keyboard and mouse. The text reads 'code cat is on the job', lolcats-style (CODE CAT)
Thinking things through out-loud here, mostly, as I think at the moment [personal profile] adevyish is the only other person with a real stake in this project (though if you are intrepid Kobo ereader owner who would like to easily browse and read fic on your device, you may too be interested!). There's basically two main challenges in this project - the first is that AO3 is designed for web, and only web, with the 'download this fic' button being the only way to access fic data in a different format, and the other is that Kobo firmware is basically a very minimal Linux OS, with no prebuilt web engine binary.

For issue two, the two solutions are 'write something to parse HTML and render it to an image' or 'use an HTML renderer someone else has written'. If I absolutely HAD to, I could do the first bit in Python - I've used the image-rendering packages in Python before, and I've even dealt with low-level interfaces for eink frame buffers. But, frankly, that's a LOT of work. Like we take reflowing text for granted, and it's only when you have to specify every linebreak yourself that you realize the number of calculations that go into it. Which leads us to 'use someone else's renderer', and for that, there are really three options: KoReader, which is written in C and Lua; Plato, which is written in Rust; and the Webkit engine that shipped with Qt4, which is C++. I actually tried the last option first, because I have a bit of experience with Qt from undergrad, but it turns out crosscompiling it is a huuuuuge pain in the ass and despite many hours attempting it I still haven't been able to succesfully crosscompile even sample code. Which leaves KoReader and Plato, and as I was going to be about equally lost in the code (as these are both single programs that happen to be OSS, rather than a general-purpose framework like Qt, and thus have basically no code documentation), I opted to go with Plato because I think going forward in my life, Rust will be handier, and Plato's GUI is nicer/more polished.

So - we've got Plato for handling the HTML -> framebuffer rendering, and for all the GUI components, which leaves me to actually write the solution to problem one, fetching data from AO3 and rendering it into a format that is ereader-friendly, which basically involves fetching pages from AO3, parsing and scraping them for particular bits of content, and then feeding that cleaned content into Plato's HTML rendering engine. And then providing GUI elements that translate on the backend to specific requests to AO3, like posting a comment, leaving kudos, bookmarking a fic, etc. I really wish there were an API, because scraping and trying to mimic web traffic this way is a lot more brittle, but lol that's never gonna happen.

I went back and forth a bit in my head about whether I should be writing this as like, an app within Plato, or as a more complete fork, and I'm leaning towards fork, because it won't be possible for stuff from this to exist in the same views as local documents (as I explicitly do not want to save these requests as local documents - the data will only ever exist in memory, like it does in a browser), so I'm going to have to write my own 'Home' interface anyway. I'm kind of thinking of this in terms of views that need to be written, and what needs to be done for each one.

Work view
- The minimum viable product, to use business-speak terms, and the bit I've actually started on
- Still struggling to decide if I want to fetch the whole work at once, or individual chapters. Drawbacks to whole-work: will have to modify the document tree to add relative anchor links to chapter titles, no easy access to per-chapter comments. Drawbacks to single-chapter: have to rewrite Plato's chapter functionality to handle remote locations rather than relative ones, have to figure out a nice way to move to the previous/next chapter in the reader when they're not part of the same document.
- I'm trying to figure out where to put access to AO3-specific functionality - the work metadata, kudos, comments, etc etc. I'd like to include it as buttons on the overlay controls, but there is already quite a lot crammed in there already. I definitely don't want to show the work metadata on every chapter the way the web interface does, because on a limited screen size that would get obnoxious so very fast to page past.
- Maybe display controls should be moved into the dropdown? As you are substantially less likely to change them per-work than you are with other docs where the base formatting may vary.

List view
- Basically any view of a list of works - a tag view, search results, an author's works, etc.
- How the fuck am I going to deal with wall-of-tags/variable lengths of summaries in such a limited space, I don't know
- I don't think I can fully proxy the advanced search page, because the auto-complete relies on XHR requests and I think the lag for making the requests and redrawing the UI is going to be too big. The sort-and-filter sidebar view is doable though because that data is populated on page load.
- Actions and metadata will vary based on the view, and go either at the top or bottom of the screens - space is somewhat less of an issue here because there'll be fewer display controls.
- Have to figure out how to handle pagination smoothly, because the AO3 results page size and the Kobo screen view page size will not be remotely the same, and will need to be tracked seperately in the backend.

Home view
- The starting point for the app. Should handle login/logout functionality, and have paths to account view stuff, and browse.
- Going to replicate the favorite tag view, but ideally I'd like to add logic to be able to favorite ANY list view. Particularly filtered search views, so I don't have to keep resetting the filters to remove results in languages I cannot read. And AO3 refuses to add this functionality themselves, so.


There's probably also some assorted smaller views (like fandom browse), but I figure I'll get to those when I need them. When will this all get done? Who knows! But the idea won't leave me alone and reading fic on my phone in bed is not great for my eyes, my shoulders, or my sleep cycle, so I have incentive. And getting to the point of 'load HTML from memory, not from disk' was a major breakthrough so I have motivation to work on this.

Profile

momijizukamori: Green icon with white text - 'I do believe in phosphorylation! I do!' with a string of DNA basepairs on the bottom (Default)
Cocoa

April 2025

M T W T F S S
 123456
78910111213
14 151617181920
21222324252627
282930    

the greatest secret of all times:

Custom text with html!Custom text with [personal profile] momijizukamori - a user name.italics! underline! strikethrough
I am a blockquote

Expand Cut Tags

No cut tags

Style Credit