Real-image posters: match the box to the poster's shape and crop the remainder

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

SPECS.md

@@ -64,7 +64,7 @@ Notes:
   - Global speed / pause state in the status bar. Pause/resume and delete a SABnzbd job.
 - **Status bar / help:** the bottom bar lists the keys that work in the current view or panel; the top right shows the connection state of each service; `?` opens the full key reference.
 - **Posters:** the detail panes show the poster at the top right; the head (title, facts, path) and the plot are one text that flows along the poster's left side and continues at full width below it (text is wrapped by arr-tui itself, with a narrower width for the first lines); on panes too small for the poster, the text simply uses the full width. Two ways to draw the poster, chosen by `ui.cover_style`:
-  - **Real images** (`kitty`, and `auto` in kitty/Ghostty): the Kitty graphics protocol with Unicode placeholders. The poster (kept up to 640 px tall) is uploaded once as PNG (`a=t,f=100,q=2`, 4096-byte base64 chunks), placed virtually with the cell footprint (`a=p,U=1,c=,r=`; a new size deletes and recreates the placement), and shown by writing U+10EEEE placeholder characters whose combining marks give row and column (the 297-entry table from the protocol) and whose foreground color is the image id. Since it is ordinary text, popups, dialogs and view changes cover it correctly and tcell's diffing keeps it in place. The footprint uses the real cell size in pixels (from the terminal's window size) so the poster is not stretched. Writes to the terminal go through the screen's tty on the UI goroutine, so they cannot interleave with drawing. Requires 24-bit color; otherwise, and inside tmux/screen, the block fallback is used. Terminals are recognised from `TERM`/`TERM_PROGRAM`/`KITTY_WINDOW_ID`/`GHOSTTY_RESOURCES_DIR`. The go-termimg library was looked at and not used: its placeholder support is marked experimental, its diacritic table does not match the protocol's, and it writes images straight to the terminal instead of through the screen.
+  - **Real images** (`kitty`, and `auto` in kitty/Ghostty): the Kitty graphics protocol with Unicode placeholders. The poster (kept up to 640 px tall) is uploaded once as PNG (`a=t,f=100,q=2`, 4096-byte base64 chunks), placed virtually with the cell footprint (`a=p,U=1,c=,r=`; a new size deletes and recreates the placement), and shown by writing U+10EEEE placeholder characters whose combining marks give row and column (the 297-entry table from the protocol) and whose foreground color is the image id. Since it is ordinary text, popups, dialogs and view changes cover it correctly and tcell's diffing keeps it in place. The footprint uses the real cell size in pixels (from the terminal's window size) and a box whose shape is within 3% of the poster's; the remaining sliver is cropped from the poster's edges before upload (one upload per size, the previous one is deleted), because the terminal letterboxes an image that does not fit its box exactly (visible grey bars beside the poster). Writes to the terminal go through the screen's tty on the UI goroutine, so they cannot interleave with drawing. Requires 24-bit color; otherwise, and inside tmux/screen, the block fallback is used. Terminals are recognised from `TERM`/`TERM_PROGRAM`/`KITTY_WINDOW_ID`/`GHOSTTY_RESOURCES_DIR`. The go-termimg library was looked at and not used: its placeholder support is marked experimental, its diacritic table does not match the protocol's, and it writes images straight to the terminal instead of through the screen.
   - **Block characters** (`quadrant`, `sextant`, and the fallback): every cell shows several pixels in the best two-color split (`quadrant`: 2×2 pixels, glyphs from the Block Elements range every font has; `sextant`: 2×3 pixels, sharper, glyphs from Symbols for Legacy Computing). `auto` picks sextants on kitty, WezTerm, foot and Ghostty, and on other terminals when fontconfig lists a font with the sextant characters (`fc-list :charset=1fb00`, about 20 ms at startup; terminals draw glyphs their font lacks from fallback fonts; skipped over SSH because the installed fonts are the remote machine's), and quadrants otherwise.
   Posters are JPEG/PNG, decoded and downscaled in the process. Library items fetch `/MediaCover/<id>/poster-250.jpg` from their Radarr/Sonarr instance with the API key; search results fall back to the provider's `remoteUrl` (fetched without credentials). Posters are cached in memory (the terminal's copies are deleted when the cache is dropped and on exit), failed downloads are not retried for two minutes, and the poster shrinks or disappears when the pane is too short. `ui.covers: false` disables it.
 - **Calendar view (F5):** a two-week window (starting today, moved a week at a time with `[`/`]`, `t` returns to today) with the library's movie releases and airing episodes, grouped by day. Radarr `GET /calendar` supplies the cinema, digital and physical release dates (taken as written: midnight UTC means that calendar day) and Sonarr `GET /calendar?includeSeries=true` the episodes with their air time (shown in local time). Unmonitored items are included with `u`. A movie appears once per release date in the window. Each entry shows type, title, episode or release kind and status (downloaded / missing / upcoming); the detail pane shows poster and plot; `Enter`/`i` starts the release search and `s` an automatic search.

internal/ui/cover.go

@@ -294,17 +294,47 @@ func (c *coverView) aspect() (w, h int) {
 	return b.Dx(), b.Dy()
 }
 
-// coverCells returns how many columns and rows a poster of the given aspect
+// coverFitTolerance is how far the shape of the cell box may differ from the
+// poster's, as a fraction. The box is chosen to match the poster's shape
+// closely; the small remainder is cropped away from the poster before it is
+// shown as a real image, because a real image placed in a box of another shape
+// is letterboxed (visible bars beside or above it).
+const coverFitTolerance = 0.03
+
+// coverCells returns how many columns and rows a poster of the given shape
 // needs inside w×h cells. cellAspect is the height of a cell divided by its
-// width, about 2 in most terminals.
+// width, about 2 in most terminals. Cell counts are whole numbers, so the
+// largest size whose box is within coverFitTolerance of the poster's shape is
+// chosen (or the closest one if none is).
 func coverCells(imgW, imgH, w, h int, cellAspect float64) (cols, rows int) {
-	cols = w
-	rows = int(float64(cols) * float64(imgH) / float64(imgW) / cellAspect)
-	if rows > h {
-		rows = h
-		cols = int(float64(rows) * cellAspect * float64(imgW) / float64(imgH))
+	if imgW <= 0 || imgH <= 0 || w <= 0 || h <= 0 || cellAspect <= 0 {
+		return 0, 0
 	}
-	return max(cols, 0), max(rows, 0)
+	shape := float64(imgW) / float64(imgH) // width / height of the poster
+	// A box of r rows is r*cellAspect*shape columns wide for the same shape.
+	maxRows := min(h, int(float64(w)/(cellAspect*shape)))
+	if maxRows < 1 {
+		return w, 0
+	}
+	bestErr := 2.0
+	for r := maxRows; r >= max(1, maxRows-6); r-- {
+		c := int(float64(r)*cellAspect*shape + 0.5)
+		if c < 1 || c > w {
+			continue
+		}
+		boxShape := float64(c) / (float64(r) * cellAspect)
+		err := boxShape/shape - 1
+		if err < 0 {
+			err = -err
+		}
+		if err <= coverFitTolerance {
+			return c, r
+		}
+		if err < bestErr {
+			bestErr, cols, rows = err, c, r
+		}
+	}
+	return cols, rows
 }
 
 func (c *coverView) Draw(screen tcell.Screen) {
@@ -341,6 +371,23 @@ func (c *coverView) Draw(screen tcell.Screen) {
 	}
 }
 
+// cropToBox returns the part of an image of the given bounds that has the same
+// shape as a box of cols×rows cells: a few pixels are trimmed from both sides
+// of the longer dimension (at most the fit tolerance), so the image fills the
+// box exactly and the terminal does not letterbox it.
+func cropToBox(b image.Rectangle, cols, rows int, cellAspect float64) image.Rectangle {
+	box := float64(cols) / (float64(rows) * cellAspect) // width / height of the box
+	shape := float64(b.Dx()) / float64(b.Dy())
+	if box < shape { // box is narrower: trim the sides
+		w := int(float64(b.Dy())*box + 0.5)
+		trim := (b.Dx() - w) / 2
+		return image.Rect(b.Min.X+trim, b.Min.Y, b.Min.X+trim+w, b.Max.Y)
+	}
+	h := int(float64(b.Dx())/box + 0.5) // box is wider: trim top and bottom
+	trim := (b.Dy() - h) / 2
+	return image.Rect(b.Min.X, b.Min.Y+trim, b.Max.X, b.Min.Y+trim+h)
+}
+
 // drawGraphics shows the poster as a real image through the Kitty graphics
 // protocol: the image is uploaded once and the cells get placeholder
 // characters. It reports false when that is not possible (no protocol, no true
@@ -350,7 +397,7 @@ func (c *coverView) drawGraphics(screen tcell.Screen, x, y, cols, rows int) bool
 	if c.gfx == nil || screen.Colors() < 1<<24 || cols > maxPlaceholders || rows > maxPlaceholders {
 		return false
 	}
-	id, err := c.gfx.imageID(c.img)
+	id, err := c.gfx.imageID(c.img, cropToBox(c.img.Bounds(), cols, rows, c.aspectOfCells()))
 	if err == nil {
 		err = c.gfx.place(id, cols, rows)
 	}

internal/ui/cover_test.go

@@ -113,8 +113,9 @@ func TestCoverCells(t *testing.T) {
 	if c, r := coverCells(40, 60, 30, 10, 2); c != 13 || r != 10 {
 		t.Errorf("tall limit: %dx%d", c, r)
 	}
-	// In a narrow box the width limits it.
-	if c, r := coverCells(40, 60, 10, 20, 2); c != 10 || r != 7 {
+	// In a narrow box the width limits it, and the box keeps the poster's shape
+	// (8×6 cells is exactly 2:3) instead of 10×7 which would leave bars.
+	if c, r := coverCells(40, 60, 10, 20, 2); c != 8 || r != 6 {
 		t.Errorf("narrow limit: %dx%d", c, r)
 	}
 }
@@ -276,3 +277,28 @@ func TestCoverStyleFor(t *testing.T) {
 		}
 	}
 }
+
+func TestCoverCellsKeepTheShape(t *testing.T) {
+	// For many poster shapes, cell shapes and boxes the chosen box must match
+	// the poster within the tolerance (or be the closest available).
+	for _, img := range [][2]int{{167, 250}, {500, 750}, {2, 3}, {1000, 1500}, {680, 1000}} {
+		for _, aspect := range []float64{1.9, 2, 2.1, 2.25} {
+			for w := 12; w <= 40; w += 7 {
+				for h := 8; h <= 30; h += 11 {
+					c, r := coverCells(img[0], img[1], w, h, aspect)
+					if c < 1 || r < 1 || c > w || r > h {
+						t.Fatalf("%v aspect %v in %dx%d: %dx%d out of bounds", img, aspect, w, h, c, r)
+					}
+					shape := float64(img[0]) / float64(img[1])
+					err := float64(c)/(float64(r)*aspect)/shape - 1
+					if err < 0 {
+						err = -err
+					}
+					if err > 0.10 { // the tolerance is 3%; tiny boxes may only get within a cell
+						t.Errorf("%v aspect %v in %dx%d: box %dx%d is %.0f%% off", img, aspect, w, h, c, r, err*100)
+					}
+				}
+			}
+		}
+	}
+}

internal/ui/kitty.go

@@ -30,12 +30,19 @@ const kittyChunk = 4096 // bytes of base64 per escape sequence, the protocol max
 type kittyGraphics struct {
 	write  func([]byte) error // writes to the terminal, in step with the screen
 	nextID uint32
-	ids    map[*image.RGBA]uint32
-	placed map[uint32][2]int // columns and rows each image is currently placed with
+	ids    map[cropKey]uint32
+	latest map[*image.RGBA]cropKey // the crop of each image that is currently uploaded
+	placed map[uint32][2]int       // columns and rows each image is currently placed with
+}
+
+// cropKey identifies an uploaded image: a poster and the part of it that was sent.
+type cropKey struct {
+	img  *image.RGBA
+	rect image.Rectangle
 }
 
 func newKittyGraphics(write func([]byte) error) *kittyGraphics {
-	return &kittyGraphics{write: write, nextID: 1, ids: map[*image.RGBA]uint32{}, placed: map[uint32][2]int{}}
+	return &kittyGraphics{write: write, nextID: 1, ids: map[cropKey]uint32{}, latest: map[*image.RGBA]cropKey{}, placed: map[uint32][2]int{}}
 }
 
 // kittyAPC builds one graphics command: ESC _ G <keys> ; <payload> ESC \.
@@ -51,14 +58,17 @@ func kittyAPC(keys string, payload []byte) []byte {
 	return b.Bytes()
 }
 
-// imageID returns the terminal's id for the image, uploading it on first use.
-func (k *kittyGraphics) imageID(img *image.RGBA) (uint32, error) {
-	if id, ok := k.ids[img]; ok {
+// imageID returns the terminal's id for the part rect of the image,
+// uploading it on first use. When the same image is needed with another crop
+// (after a resize) the earlier upload is removed.
+func (k *kittyGraphics) imageID(img *image.RGBA, rect image.Rectangle) (uint32, error) {
+	key := cropKey{img, rect}
+	if id, ok := k.ids[key]; ok {
 		return id, nil
 	}
 	var data bytes.Buffer
 	enc := png.Encoder{CompressionLevel: png.BestSpeed}
-	if err := enc.Encode(&data, img); err != nil {
+	if err := enc.Encode(&data, img.SubImage(rect)); err != nil {
 		return 0, err
 	}
 	encoded := base64.StdEncoding.EncodeToString(data.Bytes())
@@ -81,7 +91,15 @@ func (k *kittyGraphics) imageID(img *image.RGBA) (uint32, error) {
 		rest = rest[n:]
 	}
 	k.nextID++
-	k.ids[img] = id
+	if old, ok := k.latest[img]; ok {
+		if oldID, ok := k.ids[old]; ok {
+			_ = k.write(kittyAPC(fmt.Sprintf("a=d,d=I,i=%d,q=2", oldID), nil))
+			delete(k.placed, oldID)
+			delete(k.ids, old)
+		}
+	}
+	k.ids[key] = id
+	k.latest[img] = key
 	return id, nil
 }
 
@@ -110,7 +128,8 @@ func (k *kittyGraphics) deleteAll() {
 	for _, id := range k.ids {
 		_ = k.write(kittyAPC(fmt.Sprintf("a=d,d=I,i=%d,q=2", id), nil))
 	}
-	k.ids = map[*image.RGBA]uint32{}
+	k.ids = map[cropKey]uint32{}
+	k.latest = map[*image.RGBA]cropKey{}
 	k.placed = map[uint32][2]int{}
 }
 

internal/ui/kitty_test.go

@@ -5,6 +5,7 @@ import (
 	"encoding/base64"
 	"image"
 	"image/png"
+	"math"
 	"math/rand"
 	"regexp"
 	"strings"
@@ -49,7 +50,7 @@ func TestKittyUploadChunksAndEncodes(t *testing.T) {
 	k := newKittyGraphics(rec.write)
 	img := noisy(200, 300) // incompressible: far more than one 4096-byte chunk of base64
 
-	id, err := k.imageID(img)
+	id, err := k.imageID(img, img.Bounds())
 	if err != nil || id != 1 {
 		t.Fatalf("id=%d err=%v", id, err)
 	}
@@ -94,10 +95,10 @@ func TestKittyUploadChunksAndEncodes(t *testing.T) {
 
 	// The same image is not uploaded twice; another one gets the next id.
 	n := len(rec.cmds)
-	if again, _ := k.imageID(img); again != id || len(rec.cmds) != n {
+	if again, _ := k.imageID(img, img.Bounds()); again != id || len(rec.cmds) != n {
 		t.Error("an uploaded image must be reused")
 	}
-	if other, _ := k.imageID(noisy(4, 4)); other != 2 {
+	if other, _ := k.imageID(noisy(4, 4), image.Rect(0, 0, 4, 4)); other != 2 {
 		t.Errorf("second image id = %d", other)
 	}
 }
@@ -121,7 +122,8 @@ func TestKittyPlacement(t *testing.T) {
 	}
 
 	rec.cmds = nil
-	id, _ := k.imageID(noisy(2, 2))
+	small := noisy(2, 2)
+	id, _ := k.imageID(small, small.Bounds())
 	k.deleteAll()
 	if last := rec.cmds[len(rec.cmds)-1]; last != "\x1b_Ga=d,d=I,i="+itoa(int(id))+",q=2\x1b\\" {
 		t.Errorf("deleteAll sent %q", last)
@@ -237,7 +239,39 @@ func TestCoverViewGraphicsDrawsPlaceholders(t *testing.T) {
 	// A cell aspect of 2.5 (taller cells) changes the footprint of the same box.
 	c.cellAspect = func() float64 { return 2.5 }
 	c.Draw(screen)
-	if len(rec.cmds) != 4 || !strings.Contains(rec.cmds[3], "c=16,r=10") {
-		t.Errorf("after the cell aspect changed: %q", rec.cmds[2:])
+	// The box is now 17×10 cells, a different shape: a new crop is uploaded, the
+	// old upload is removed, and the image is placed afresh.
+	var placements []string
+	for _, cmd := range rec.cmds[2:] {
+		placements = append(placements, cmd[:min(len(cmd), 30)])
+	}
+	last := rec.cmds[len(rec.cmds)-1]
+	if !strings.Contains(last, "c=17,r=10") || !strings.Contains(strings.Join(rec.cmds[2:], ""), "a=d,d=I,i=1") {
+		t.Errorf("after the cell aspect changed: %q", placements)
+	}
+}
+
+func TestCropToBox(t *testing.T) {
+	b := image.Rect(0, 0, 167, 250) // a 2:3 poster
+	// A box slightly narrower than the poster: the sides are trimmed.
+	got := cropToBox(b, 17, 13, 2.1) // box shape 17/(13*2.1) = 0.6227 < 0.668
+	if got.Dy() != 250 || got.Dx() >= 167 || got.Min.X != (167-got.Dx())/2 {
+		t.Errorf("narrow box: %v", got)
+	}
+	if shape := float64(got.Dx()) / float64(got.Dy()); math.Abs(shape-17.0/(13*2.1)) > 0.005 {
+		t.Errorf("cropped shape %.4f, want %.4f", shape, 17.0/(13*2.1))
+	}
+	// A box slightly wider: top and bottom are trimmed.
+	got = cropToBox(b, 18, 12, 2.1) // 18/(12*2.1) = 0.714 > 0.668
+	if got.Dx() != 167 || got.Dy() >= 250 || got.Min.Y != (250-got.Dy())/2 {
+		t.Errorf("wide box: %v", got)
+	}
+	// A box of exactly the poster's shape changes nothing.
+	if got := cropToBox(image.Rect(0, 0, 40, 60), 20, 15, 2); got != image.Rect(0, 0, 40, 60) {
+		t.Errorf("exact box: %v", got)
+	}
+	// Offsets of sub-images are respected.
+	if got := cropToBox(image.Rect(10, 20, 50, 80), 18, 10, 2); got.Min.X != 10 || got.Max.X != 50 || got.Min.Y < 20 || got.Max.Y > 80 || got.Dy() >= 60 {
+		t.Errorf("offset image: %v", got)
 	}
 }