Abstract: Traditional multimodal reasoning methods have problems such as insufficient processing of image data, ignoring local information, solely utilizing multimodal information, and ignoring the complex interactions between different modalities. Based on this, an information fusion knowledge graph inference model AGRMCG based on CNN and GNN is proposed. By introducing a new multi-scale attention module on VGG16, the obtained feature representations will be dynamically adjusted according to the influence of attention to enhance the model's ability to represent images. Simultaneously utilizing local structure encoders and global information encoders to extract knowledge graph entity features from different perspectives, jointly enhancing the ability to capture graph structure information. Finally, in terms of information fusion, multimodal fusion reasoning is carried out in two stages. In the first stage of multimodal fusion, Tucker decomposition and contrastive learning are used to capture complex feature information in multimodal data. The context model is used to obtain the prediction scores of each modality and the fused multimodal data. In the second stage of decision fusion, the predictions of all modalities, including multimodal data, are weighted and averaged to enhance the complementarity of information from different modalities and obtain better inference results. The extensive experimental results conducted on three datasets ultimately showed that the AGRMCG model can effectively alleviate the current problem of poor performance in multimodal feature extraction and information fusion. Compared with six benchmark models, the proposed model improved the prediction accuracy by an average of 3.168%.