In the context of target detection under infrared conditions for drones, the common issues of high missed detection rates, low signal-to-noise ratio, and blurred edge features for small targets are prevalent. To address these challenges, this paper proposes an improved detection algorithm based on YOLOv11n. First, a Dynamic Multi-Scale Feature Fusion and Adaptive Weighting approach is employed to design an Adaptive Focused Diffusion Pyramid Network (AFDPN), which enhances the feature expression and transmission capability of shallow small targets, thereby reducing the loss of detailed information. Then, combined with an Edge Enhancement (EE) module, the model improves the extraction of infrared small target edge features through low-frequency suppression and high-frequency enhancement strategies. Experimental results on the publicly available HIT-UAV dataset show that the improved model achieves a 3.8% increase in average detection accuracy and a 3.0% improvement in recall rate compared to YOLOv11n, with a computational cost of only 9.1 GFLOPS. In comparison experiments, the detection accuracy and model size balance achieved the optimal solution, meeting the lightweight deployment requirements for drone-based systems. This method provides a high-precision, lightweight solution for small target detection in drone-based infrared imagery.