Addressing challenges in accurately detecting persimmon fruit quality in orchards—such as reliance on manual grading, low efficiency, severe foliage obstruction, and subtle differences between quality grades—this paper proposes a lightweight persimmon detection model based on an improved YOLOv8s architecture. First, the Conv layer in the backbone network is replaced with an ADown module to reduce model complexity. Second, MSFAN is introduced in the Neck layer to fully extract texture features from persimmon images, highlighting differences between quality grades. Finally, the Wise-IoU loss function mitigates the impact of low-quality sample data on grading accuracy. The improved model accurately identifies and separates persimmons of varying quality, effectively addressing quality grading detection in complex backgrounds. This provides a viable technical approach for achieving persimmon quality grading detection.