Abstract: To address the challenges of feature extraction and localization in small object detection, this paper proposes an improved YOLO11 algorithm, termed PSSM-YOLO, specifically designed for small object detection tasks. The method adopts YOLO11n as the baseline, incorporates the C2f structure from YOLOv8, and integrates a Progressive Scale-aware Spatial Modeling (PSSM) module. Through multi-branch feature fusion and multi-receptive-field channel-wise convolution, the approach effectively preserves fine-grained information while incorporating contextual cues, thereby enhancing the model's feature representation capability for small objects. Furthermore, a C2PSA-DA module integrating deformable attention mechanism is designed, which introduces a dynamic sampling strategy to adaptively focus on critical target regions and suppress redundant background, thus strengthening the network's feature extraction capability for small objects and their contextual information in complex traffic scenes. A P2 detection layer with 1/4 downsampling is newly added to the Neck, enabling shallow fine-grained features to directly participate in prediction and reducing the loss of small object information caused by multiple downsampling operations, thereby further improving detection performance. In addition, the WIoU v3 loss function is introduced, which employs a dynamic non-monotonic focusing mechanism to reinforce the learning of medium-quality samples while suppressing the interference from low-quality samples. Experimental results on the CCTSDB2021 dataset demonstrate that PSSM-YOLO achieves significant improvements in overall detection accuracy and robustness, outperforming the baseline YOLO11n by 4.5% in mAP@50 and 3.6% in mAP@50:95.