diff --git a/CHANGELOG.md b/CHANGELOG.md index d37a59e..8fdcb2f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,10 @@ ## Unreleased +- **Trakt lists show faster after a restart:** the Radarr/Sonarr lookups of their titles are kept + on disk too (3 days, `lookups.json`), so a list takes one request to Trakt and a quick library + check per title instead of a lookup (about a second each) per title. Whether a title is in the + library is always asked afresh. - **Posters of titles not in the library load faster:** they are fetched from TheMovieDB at the size shown (780 pixels wide for real images) instead of the original Radarr names, which is often 2000×3000 and over 2 MB: about a tenth of the download and a sixth of the decoding time. diff --git a/README.md b/README.md index 7b0f4c7..4227172 100644 --- a/README.md +++ b/README.md @@ -14,7 +14,8 @@ A terminal UI to search for movies and series with [Radarr](https://radarr.video ticked, the others can be added right there (Enter). Needs a free Trakt client ID (see Configure). - **Library**: movies and series with their status; the dot before a title turns blue while it is downloading, and the details say how far. -- **Cache**: posters (30 days, at most 200 MB) and casts (7 days) are kept between runs in +- **Cache**: posters (30 days, at most 200 MB), casts (7 days) and the Radarr/Sonarr lookups of + Trakt list titles (3 days) are kept between runs in `~/.cache/arr-tui` (`$XDG_CACHE_HOME`; `~/Library/Caches/arr-tui` on macOS), so they show at once after a restart. Deleting the folder is always safe. - **Season posters**: in a series' seasons and episodes, the poster follows the selected season. diff --git a/SPECS.md b/SPECS.md index 1a1a64a..e278375 100644 --- a/SPECS.md +++ b/SPECS.md @@ -71,7 +71,7 @@ Notes: - **Download notifications:** a background poller (independent of the visible view) reads SABnzbd's history every 15 s and announces downloads that reach `Completed` or `Failed` as a popup and log entry; the first poll only records the current state. `ui.notify_downloads` (default true) and `ui.bell` (default false, rings the terminal bell) control it. Connection errors are not reported here, the indicators and the log already do that. - **Trakt lists:** while the search box of Movies or Series is empty, the list shows one of Trakt's public lists instead of search results: trending now (`/movies|shows/trending`, "N watching"), most watched this week (`/…/watched/weekly`, "N watched it"), anticipated (`/…/anticipated`, "on N lists"), popular (`/…/popular`), and for movies the US box office of last weekend (`/movies/boxoffice`, "$N last weekend"); 20 titles each (`page=1&limit=20`), with the headers `trakt-api-key` (the client ID, `trakt.client_id` or `ARR_TUI_TRAKT_CLIENT_ID`), `trakt-api-version: 2`, `Content-Type` and `User-Agent`. Every entry is looked up in Radarr (`tmdb:`) or Sonarr (`tvdb:`), at most four at a time, so it behaves like a search result (details, poster, tick when in the library, add, releases); entries without that id or unknown to the service are left out, in list order otherwise. `[` `]` switch lists (also from the empty search box), `r` reloads; a fetched list is shown again for 15 minutes without asking. Without a client ID the list says how to get one. Changing the client ID or a service in the Config view drops the fetched lists. `ARR_TUI_TRAKT_URL` points the client at another server (used by `dev/screenshots`). - **Downloading in the Library:** while the Library is visible it reads the Radarr and Sonarr queues (`/queue`, matched to titles by `movieId` / `seriesId`) when it opens, every 10 s, and 3 s after a grab. A title with queue entries gets a blue mark (filled when monitored, hollow when not) instead of the green or grey one, and its details a line "Downloading 45% · 0:06:12 left" (series: "Downloading N episodes"). A queue that cannot be read leaves the marks as they were. After a successful grab the release list closes (if nothing was opened on top of it). -- **Disk cache:** posters and casts are kept between runs in `os.UserCacheDir()/arr-tui` (`$XDG_CACHE_HOME` or `~/.cache` on Linux, `~/Library/Caches` on macOS; directories `0700`, files written atomically). Posters are stored as decoded (at most 640 or 160 pixels tall, JPEG quality 90), one file per poster and size (small copy for block characters, large for real images) under `posters-v2/.img` (the folder is versioned: folders of earlier storage formats are deleted at startup, so their files are not used as they are); a file is used for 30 days (by its modification time) and dropped when it cannot be decoded; at startup files older than that are deleted, then the oldest until the rest fit 200 MB. Casts are one `people.json` (key = `movie:tmdb:` / `show:tvdb:`, with the time it was fetched); entries are used for 7 days, at most 2000 are kept (the oldest go first), titles Trakt does not know are kept as having no cast; new casts are written 5 s after the last one and when the program quits. Changing the Trakt client ID keeps the casts and only forgets failed lookups. +- **Disk cache:** posters and casts are kept between runs in `os.UserCacheDir()/arr-tui` (`$XDG_CACHE_HOME` or `~/.cache` on Linux, `~/Library/Caches` on macOS; directories `0700`, files written atomically). Posters are stored as decoded (at most 640 or 160 pixels tall, JPEG quality 90), one file per poster and size (small copy for block characters, large for real images) under `posters-v2/.img` (the folder is versioned: folders of earlier storage formats are deleted at startup, so their files are not used as they are); a file is used for 30 days (by its modification time) and dropped when it cannot be decoded; at startup files older than that are deleted, then the oldest until the rest fit 200 MB. Casts are one `people.json` (key = `movie:tmdb:` / `show:tvdb:`, with the time it was fetched); entries are used for 7 days, at most 2000 are kept (the oldest go first), titles Trakt does not know are kept as having no cast; new casts are written 5 s after the last one and when the program quits. Changing the Trakt client ID keeps the casts and only forgets failed lookups. Trakt list titles: whether a title is in the library is asked every time (`GET movie?tmdbId=` / `series?tvdbId=`, a quick query of the server's own database; the answer is checked against the id, as a version that ignores the filter sends the whole library); for titles that are not, the Radarr/Sonarr lookup (which asks the metadata provider, about a second each) is kept in `lookups.json` (key = `tmdb:` / `tvdb:`, the result as the server sent it, with the time it was fetched) for 3 days, at most 2000, written like the casts. The Trakt list itself is fetched again after 15 minutes as before. - **Season posters:** the series drill-down reads the series with `includeSeasonImages=true` (Sonarr only sends season posters when asked, and keeps no copy of them: they are TheTVDB links, fetched without credentials). A selected season or episode shows its season's poster; seasons without one, or whose poster cannot be loaded, show the series poster. - **Cast:** with a Trakt client ID, every details pane (search results, Trakt lists, Library, Calendar, the series drill-down) shows the director(s) of a movie or the "created by" credits of a series and up to 8 cast members with their characters below the plot ("and N more" after that). The title is found on Trakt by its TMDb id (movies) or TVDB id (series), else IMDb id (`/search//?type=movie|show`), then `/movies|shows//people` is read. Casts are cached, also on disk (see Disk cache; failures are retried after two minutes) and, like posters, only loaded once the selection has rested for 120 ms, so a held arrow key does not spend Trakt's allowance (500 requests per five minutes). - **Config view (F6):** one list with a field for every setting of the config file, one below the other under the headings Radarr, Sonarr, SABnzbd, Trakt and Interface (a single scrolling form; ↑↓ passes over the headings and ↓ after the last field reaches the Save / Revert buttons, so no Tab is needed; Tab jumps between the fields and the buttons). The `Test` button is part of the list, right below the SABnzbd settings; `Save` and `Revert` are centered in a bar below the list. When the config file does not exist, it is created empty (`config.Create`: directory `0700`, file `0600`) and the program opens on this view. A service with neither address nor key is "not configured": allowed by validation (half-filled ones are not), its client answers every call with `ErrNotConfigured` without a request, its indicator stays grey (logged once as information, not as unreachable) and its lists say "not configured (Config view, F6)" instead of an error. The fields start from the settings in use; API keys are masked (a checkbox shows them). The default quality profile and root folder of Radarr and Sonarr are dropdowns filled from the server (`/qualityprofile`, `/rootfolder`) with the address and key currently in the fields: when the view opens, on Revert, and 700 ms after the address or key was last changed; answers for values that have changed since are dropped, a failure is reported once in the result box. "(none)" comes first; a configured value the server does not list stays as an option, marked "(not on the server)", so opening the view never changes a setting. `Test` builds temporary clients from the typed values and reports each service. `Save` validates like startup does (URLs, non-empty keys, refresh 1–3600 s), then writes the file from the file's own contents plus the edits (`config.LoadFile`), so values that only come from `ARR_TUI_*` environment variables are left as the file has them and are shown greyed out; the write is atomic (temp file + rename) and keeps the file's permissions (`0600` when new); defaults such as `covers: true` stay out of the file unless it already named them. Everything then applies without a restart: the Radarr/Sonarr/SABnzbd clients are re-pointed (`Reconfigure`, guarded by a mutex), the connection indicators, add-form defaults and options are refreshed, the library and calendar reload, the Downloads interval, notifications, bell and poster style are switched live. diff --git a/internal/radarr/radarr.go b/internal/radarr/radarr.go index 6fd3d64..b8ceedc 100644 --- a/internal/radarr/radarr.go +++ b/internal/radarr/radarr.go @@ -102,6 +102,19 @@ func (c *Client) Movies(ctx context.Context) ([]Movie, error) { return movies, err } +// MovieByTMDB returns the library movie with the TMDb id, ok=false if the +// library has none. Unlike a lookup it does not ask the metadata provider. +func (c *Client) MovieByTMDB(ctx context.Context, tmdbID int) (Movie, bool, error) { + var movies []Movie + err := c.Do(ctx, http.MethodGet, "movie", url.Values{"tmdbId": {strconv.Itoa(tmdbID)}}, nil, &movies) + for _, m := range movies { // checked: a version that ignores the filter sends the whole library + if m.TMDBID == tmdbID && m.InLibrary() { + return m, true, err + } + } + return Movie{}, false, err +} + // AddOptions are the choices from the add form. type AddOptions struct { QualityProfileID int diff --git a/internal/sonarr/sonarr.go b/internal/sonarr/sonarr.go index 6006766..c3bbe7f 100644 --- a/internal/sonarr/sonarr.go +++ b/internal/sonarr/sonarr.go @@ -111,6 +111,19 @@ func (c *Client) Series(ctx context.Context) ([]Series, error) { return series, err } +// SeriesByTVDB returns the library series with the TVDB id, ok=false if the +// library has none. Unlike a lookup it does not ask the metadata provider. +func (c *Client) SeriesByTVDB(ctx context.Context, tvdbID int) (Series, bool, error) { + var series []Series + err := c.Do(ctx, http.MethodGet, "series", url.Values{"tvdbId": {strconv.Itoa(tvdbID)}}, nil, &series) + for _, s := range series { // checked: a version that ignores the filter sends the whole library + if s.TVDBID == tvdbID && s.InLibrary() { + return s, true, err + } + } + return Series{}, false, err +} + // AddOptions are the choices from the add form. type AddOptions struct { QualityProfileID int diff --git a/internal/ui/app.go b/internal/ui/app.go index 7bef2f4..e4c9d3e 100644 --- a/internal/ui/app.go +++ b/internal/ui/app.go @@ -37,7 +37,8 @@ type App struct { theme theme // the colors in use trakt *trakt.Client // trending, most watched and anticipated lists, casts people *peopleCache // casts loaded from Trakt - disk *diskCache // posters and casts kept between runs + disk *diskCache // posters, casts and lookups kept between runs + lookups *lookupStore // Radarr/Sonarr lookups of Trakt list titles // Called (on the UI goroutine) a moment after a release was grabbed. afterGrab []func() // The Movies and Series views, to refresh their Trakt lists. @@ -188,8 +189,9 @@ func New(cfg *config.Config) *App { if u := os.Getenv("ARR_TUI_TRAKT_URL"); u != "" { a.trakt.SetBaseURL(u) // another Trakt API, for development (dev/screenshots) } - movies := newSearchView(a, radarrBackend{radarrClient}) - series := newSearchView(a, sonarrBackend{sonarrClient}) + a.lookups = newLookupStore(a.disk) + movies := newSearchView(a, radarrBackend{radarrClient, a.lookups}) + series := newSearchView(a, sonarrBackend{sonarrClient, a.lookups}) a.searches = []*searchView{movies, series} a.views = []view{ {key: '1', name: "Movies", body: movies.root, onShow: movies.activate, hints: searchHints}, @@ -463,6 +465,7 @@ func (a *App) Run() error { if a.people.dirty { // casts fetched in the last seconds _ = a.disk.savePeople(a.peopleSnapshot()) } + _ = a.lookups.save() }() go a.disk.prunePosters() // The screen draws through the theme (see themedScreen). diff --git a/internal/ui/backend.go b/internal/ui/backend.go index 80b4d4e..42285d6 100644 --- a/internal/ui/backend.go +++ b/internal/ui/backend.go @@ -72,7 +72,10 @@ func (m movieResult) withID(id int) result { return m } -type radarrBackend struct{ c *radarr.Client } +type radarrBackend struct { + c *radarr.Client + lookups *lookupStore +} func (b radarrBackend) name() string { return "Radarr" } func (b radarrBackend) noun() string { return "Movies" } @@ -91,14 +94,30 @@ func (b radarrBackend) search(ctx context.Context, term string) ([]result, error func (radarrBackend) lists() []trakt.List { return trakt.MovieLists } func (b radarrBackend) resolve(ctx context.Context, e trakt.Entry) (result, error) { - if e.IDs.TMDB == 0 { + id := e.IDs.TMDB + if id == 0 { return nil, errNotListed } - found, err := b.search(ctx, radarr.TMDBTerm(e.IDs.TMDB)) - if err != nil || len(found) == 0 { + if m, ok, err := b.c.MovieByTMDB(ctx, id); err != nil { + return nil, err + } else if ok { + return movieResult{Movie: m, poster: coverFor(b.c.Client, m.Images, true)}, nil + } + // Not in the library: the lookup from an earlier run, if fresh. + key := radarr.TMDBTerm(id) + var m radarr.Movie + if b.lookups.get(key, &m) { + return movieResult{Movie: m, poster: coverFor(b.c.Client, m.Images, false)}, nil + } + movies, err := b.c.Lookup(ctx, key) + if err != nil || len(movies) == 0 { return nil, err } - return found[0], nil + m = movies[0] + if !m.InLibrary() { + b.lookups.put(key, m) + } + return movieResult{Movie: m, poster: coverFor(b.c.Client, m.Images, m.InLibrary())}, nil } func (b radarrBackend) add(ctx context.Context, r result, c addChoice) (int, error) { @@ -136,7 +155,10 @@ func (s seriesResult) withID(id int) result { return s } -type sonarrBackend struct{ c *sonarr.Client } +type sonarrBackend struct { + c *sonarr.Client + lookups *lookupStore +} func (b sonarrBackend) name() string { return "Sonarr" } func (b sonarrBackend) noun() string { return "Series" } @@ -165,14 +187,30 @@ func (b sonarrBackend) search(ctx context.Context, term string) ([]result, error func (sonarrBackend) lists() []trakt.List { return trakt.ShowLists } func (b sonarrBackend) resolve(ctx context.Context, e trakt.Entry) (result, error) { - if e.IDs.TVDB == 0 { + id := e.IDs.TVDB + if id == 0 { return nil, errNotListed } - found, err := b.search(ctx, "tvdb:"+itoa(e.IDs.TVDB)) - if err != nil || len(found) == 0 { + if s, ok, err := b.c.SeriesByTVDB(ctx, id); err != nil { + return nil, err + } else if ok { + return seriesResult{Series: s, poster: coverFor(b.c.Client, s.Images, true)}, nil + } + // Not in the library: the lookup from an earlier run, if fresh. + key := "tvdb:" + itoa(id) + var s sonarr.Series + if b.lookups.get(key, &s) { + return seriesResult{Series: s, poster: coverFor(b.c.Client, s.Images, false)}, nil + } + series, err := b.c.Lookup(ctx, key) + if err != nil || len(series) == 0 { return nil, err } - return found[0], nil + s = series[0] + if !s.InLibrary() { + b.lookups.put(key, s) + } + return seriesResult{Series: s, poster: coverFor(b.c.Client, s.Images, s.InLibrary())}, nil } func (b sonarrBackend) add(ctx context.Context, r result, c addChoice) (int, error) { diff --git a/internal/ui/diskcache.go b/internal/ui/diskcache.go index 83d4c44..93c163f 100644 --- a/internal/ui/diskcache.go +++ b/internal/ui/diskcache.go @@ -169,6 +169,42 @@ func (d *diskCache) savePeople(all map[string]savedPeople) error { return writeAtomic(d.peoplePath(), data) } +func (d *diskCache) lookupsPath() string { return filepath.Join(d.dir, "lookups.json") } + +// loadLookups reads the stored Radarr/Sonarr lookups that are not too old. +func (d *diskCache) loadLookups() map[string]savedLookup { + out := map[string]savedLookup{} + if d.dir == "" { + return out + } + data, err := os.ReadFile(d.lookupsPath()) + if err != nil { + return out + } + var all map[string]savedLookup + if json.Unmarshal(data, &all) != nil { + return out // unreadable: start afresh, it is only a cache + } + for k, l := range all { + if time.Since(l.Saved) <= lookupTTL { + out[k] = l + } + } + return out +} + +// saveLookups writes the lookups. +func (d *diskCache) saveLookups(all map[string]savedLookup) error { + if d.dir == "" { + return nil + } + data, err := json.Marshal(all) + if err != nil { + return err + } + return writeAtomic(d.lookupsPath(), data) +} + // writeAtomic writes a file through a temporary file and a rename, creating // the directory (private) if needed. func writeAtomic(path string, data []byte) error { diff --git a/internal/ui/lookups.go b/internal/ui/lookups.go new file mode 100644 index 0000000..b5364e7 --- /dev/null +++ b/internal/ui/lookups.go @@ -0,0 +1,95 @@ +package ui + +import ( + "encoding/json" + "maps" + "slices" + "sync" + "time" +) + +const ( + lookupTTL = 3 * 24 * time.Hour // a stored lookup is used this long + maxLookupsSaved = 2000 // lookups kept (in memory and on disk) + lookupSaveDelay = 5 * time.Second // new lookups are written to disk this long after the last one +) + +// lookupStore keeps the Radarr/Sonarr lookups of Trakt list titles that are +// not in the library, so a list shows at once after a restart: a lookup asks +// the metadata provider and takes about a second, while whether a title is +// in the library is asked of the server every time (that is quick, and must +// not be stale). Safe for use from any goroutine; read from disk on first use. +type lookupStore struct { + disk *diskCache + + mu sync.Mutex + loaded bool + entries map[string]savedLookup + dirty bool + timer *time.Timer +} + +// savedLookup is one lookup result as Radarr/Sonarr sent it. +type savedLookup struct { + Saved time.Time `json:"saved"` + Data json.RawMessage `json:"data"` +} + +func newLookupStore(disk *diskCache) *lookupStore { return &lookupStore{disk: disk} } + +func (s *lookupStore) load() { + if !s.loaded { + s.loaded, s.entries = true, s.disk.loadLookups() + } +} + +// get decodes the stored lookup of key into v, if there is a fresh one. +func (s *lookupStore) get(key string, v any) bool { + s.mu.Lock() + defer s.mu.Unlock() + s.load() + e, ok := s.entries[key] + if !ok || time.Since(e.Saved) > lookupTTL { + return false + } + return json.Unmarshal(e.Data, v) == nil +} + +// put stores a lookup, dropping the oldest ones beyond maxLookupsSaved, and +// has it written to disk soon. +func (s *lookupStore) put(key string, v any) { + data, err := json.Marshal(v) + if err != nil { + return + } + s.mu.Lock() + defer s.mu.Unlock() + s.load() + s.entries[key] = savedLookup{Saved: time.Now(), Data: data} + if len(s.entries) > maxLookupsSaved { + keys := slices.SortedFunc(maps.Keys(s.entries), func(x, y string) int { + return s.entries[x].Saved.Compare(s.entries[y].Saved) + }) + for _, k := range keys[:len(keys)-maxLookupsSaved] { + delete(s.entries, k) + } + } + s.dirty = true + if s.timer != nil { + s.timer.Stop() + } + s.timer = time.AfterFunc(lookupSaveDelay, func() { _ = s.save() }) +} + +// save writes the lookups to disk if there are new ones. +func (s *lookupStore) save() error { + s.mu.Lock() + if !s.dirty { + s.mu.Unlock() + return nil + } + s.dirty = false + all := maps.Clone(s.entries) + s.mu.Unlock() + return s.disk.saveLookups(all) +} diff --git a/internal/ui/lookups_test.go b/internal/ui/lookups_test.go new file mode 100644 index 0000000..073204e --- /dev/null +++ b/internal/ui/lookups_test.go @@ -0,0 +1,63 @@ +package ui + +import ( + "context" + "net/http" + "net/http/httptest" + "sync/atomic" + "testing" + + "git.stinnesbeck.com/nils/arr-tui/internal/radarr" + "git.stinnesbeck.com/nils/arr-tui/internal/trakt" +) + +func TestListLookupsKeptOnDisk(t *testing.T) { + var lookups, libraryAsks atomic.Int32 + inLibrary := false + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + switch r.URL.Path { + case "/api/v3/movie": + libraryAsks.Add(1) + if inLibrary && r.URL.Query().Get("tmdbId") == "949" { + w.Write([]byte(`[{"id":7,"title":"Heat","year":1995,"tmdbId":949}]`)) + return + } + w.Write([]byte(`[{"id":3,"title":"Other","tmdbId":1}]`)) // ignores the filter + case "/api/v3/movie/lookup": + lookups.Add(1) + w.Write([]byte(`[{"title":"Heat","year":1995,"tmdbId":949,"overview":"Cops and robbers."}]`)) + } + })) + defer srv.Close() + disk := &diskCache{dir: t.TempDir()} + entry := trakt.Entry{Title: "Heat", IDs: trakt.IDs{TMDB: 949}} + resolve := func() result { + t.Helper() + store := newLookupStore(disk) // a new run + b := radarrBackend{radarr.NewClient(srv.URL, "k"), store} + r, err := b.resolve(context.Background(), entry) + if err != nil || r == nil { + t.Fatalf("resolve: %v %v", r, err) + } + if err := store.save(); err != nil { + t.Fatal(err) + } + return r + } + + if r := resolve(); r.inLibrary() || lookups.Load() != 1 { + t.Fatalf("first run: in library %v, %d lookups", r.inLibrary(), lookups.Load()) + } + r := resolve() + if lookups.Load() != 1 || r.details().plot != "Cops and robbers." { + t.Errorf("second run: %d lookups, plot %q", lookups.Load(), r.details().plot) + } + // Added meanwhile (elsewhere): the library answer wins over the stored lookup. + inLibrary = true + if r := resolve(); !r.inLibrary() || lookups.Load() != 1 { + t.Errorf("after adding: in library %v, %d lookups", r.inLibrary(), lookups.Load()) + } + if libraryAsks.Load() != 3 { + t.Errorf("library asked %d times", libraryAsks.Load()) + } +}