The inbox of a triangle
In affine geometry, the minimum-area ellipse surrounding a given triangle and the maximum-area ellipse within it (its two Steiner ellipses) are concentric and similar. This can be seen easily by performing an affine transformation to an equilateral triangle, observing that in this case these ellipses are concentric circles (the circumcircle and incircle), and that the extreme ellipses of the transformed shape are the transforms of the extreme ellipses of the original shape. In Cartesian geometry, where the Cartesian coordinates can be independently linearly transformed or swapped, something similar turns out to happen. In this case, the minimum-area axis-parallel rectangle surrounding a given triangle (its bounding box) and the maximum-area rectangle within it (let’s call it the inbox) are always similar, though not concentric.
The inbox always touches all three sides of its triangle. For if it missed any one of the sides, it could be made larger, while staying within the triangle, by a dilation centered at the opposite vertex. It turns out that for triangles in general position (no two vertex coordinates equal) there are two cases, shown above. If the two median coordinates belong to different triangle vertices (as is true for all acute triangles) then the inbox touches all three sides at separate points, and has a fourth vertex interior to the triangle, as shown on the left. If there is one triangle vertex with both median coordinates (necessarily an obtuse vertex), then the inbox touches that vertex and the opposite side, with two other vertices free, as shown on the right. Let’s call these the acute and obtuse cases, even though the triangle in the acute case might actually be obtuse.
In the obtuse case, we can apply a Cartesian transformation to make the bounding box square. The inbox lies within an isosceles right triangle formed by axis-parallel lines through the obtuse vertex and by the opposite side of the triangle. By the symmetry of this isosceles right triangle, the inbox is also square. Just as in the Steiner ellipse argument, this implies that the inbox of the untransformed triangle is similar to its bounding box.
In the acute case, let’s again draw axis-parallel lines through the median coordinates. Each line crosses one triangle side, and the line segment between these two crossings cuts the triangle into a smaller triangle (light blue below) and an orthodiagonal quadrilateral (dark blue), with the property that any rectangle that touches the three sides of the triangle also touches the fourth side of the quadrilateral. In this case, the rectangle formed by the four midpoints of the quadrilateral (its Varignon rectangle) must be the one with maximum area. For, this rectangle has exactly half the area of the orthodiagonal quadrilateral, as the four triangular flaps surrounding it can be folded over to exactly cover the rectangle. Any other rectangle would have two longer flaps and two shorter flaps, which when folded would overlap near the middle of the rectangle, showing that the total flap area is larger than the rectangle area.
The inbox has half the width and half the height of the bounding box of the orthodiagonal quadrilateral, so they both are similar. I have a coordinate-based proof that they are also both similar to the bounding box of the original triangle, but not a nice synthetic proof. Suppose we perform a Cartesian transformation of the triangle so that its cut-off vertex (the one not part of the orthodiagonal quadrilateral) is at the origin and so that its bounding box is the unit square. Let the median coordinates of the triangle vertices be \(p\) and \(q\). Then the vertex of the bounding box of the orthodiagonal quadrilateral that is closest to the origin has coordinates \((pq,pq)\), on the diagonal of the unit square, and its farthest vertex has coordinates \((1,1)\), so it is a square too.
The center of the Steiner ellipses is the centroid of the triangle, the only affine-equivariant triangle center. Both the center of the inbox and the center of its similarity with the bounding box appear to be Cartesian triangle centers in the sense of my previous post: they are continuous Cartesian-equivariant functionals from triangles to points. Unlike the centroid or the bounding-box center, they cannot be calculated separately in the two coordinates, as can be seen from the first example, where the median \(y\)-coordinate is halfway between the other two but the inbox center and center of similarity \(y\)-coordinates are not. However, they differ from the equal-box-area center of my previous post, which is outside the triangle in the obtuse case while these centers are always inside.