The rapid expansion of photovoltaic (PV) power plants has created a pressing need for efficient and reliable operation and maintenance (O&M). Traditional manual inspection is slow, costly, and prone to error, motivating the use of unmanned aerial vehicles (UAVs) with infrared and visible cameras for automated monitoring. In this paper, we propose a YOLO-based multi-task framework for simultaneous PV defect detection and hazard-level classification. We constructed a dataset of 5,000 annotated UAV images from the Riyue PV power plant, covering ten defect categories and four severity levels (LV1–LV4). To support severity grading, the YOLO architecture was extended with a dual-task head and an ordinal regression scheme. The model was trained with a compound loss combining bounding-box regression, objectness, defect classification, and hazard-level supervision. Experimental evaluation on real UAV inspection data (224 strings, 30 ground-truth defects) shows that the proposed approach achieves mAP50 of 95.6%, recall of 92.7%, and severity classification accuracy of 90.8%. The system detects both minor anomalies (e.g., bird droppings, soiling) and critical faults (e.g., missing panels, disconnections) in real time at over 40 FPS, providing actionable insights for maintenance prioritization. These results demonstrate that YOLO-based UAV inspection offers a robust and scalable solution for intelligent PV O&M.