海洋是高质量发展的要地，海洋科学大数据的发展为认知和经略海洋带来机遇的同时也引入了新的挑战。海洋科学大数据具有超多模态的显著特征，目前尚未形成面向海洋领域特色的多模态智能计算理论体系和技术框架。因此，本文首次从多模态数据技术的视角，系统性介绍面向海洋现象/过程的智能感知、认知和预知的交叉研究进展。首先，通过梳理海洋科学大数据全生命周期的阶段演进过程，明确海洋多模态智能计算的研究对象、科学问题和典型应用场景。其次，在海洋多模态大数据内容分析、推理预测和高性能计算3个典型应用场景中展开现有工作的系统性梳理和介绍。最后，针对海洋数据分布和计算模式的差异性，提出海洋多模态大数据表征建模、跨模态关联、推理预测以及高性能计算4个关键科学问题中的挑战，并提出未来展望。

The marine-oriented research is essential to high-quality of human-based development.But,the current recognition of the ocean system is less than 5%.To understand the ocean,big marine data is acquired from observation,monitoring,investigation and statistics.Thanks to the development of the multi-scaled ocean observation system,the extensive of multi-modal marine oriented data has developed via remote sensing image,spatio-temporal analysis,simulation data,literature review and video&audio monitoring.To resilient the sustainable development of human society,current deep analysis and multimodal ocean data mining method has promoted the marine understanding on the aspects of ocean dynamic processes,energy and material cycles,the evolution of blue life,scientific discovery,healthy environment,and the quick response of extreme weather and climate change.Compared to traditional big data,the multi-modal big ocean data has its unique features,such as the super-giant system (covering 71% of the earth's surface,daily increment (10 TB),super multi-perspectives ("land-sea-air-ice-earth based" coupling, "hydrometeorological-acoustical-optical and electromagnetic-based" polymorphism),super spatial scale ("centimeter to hundreds kilometer based"),and temporal scale ("micro-second to inter-decadal based").These features-derived challenges of existing multi-modal intelligent computing technology have to deal with such problems as cross-scale and multi-modal fusion analyses,multi-disciplinary and multi-domain coordinated reasoning,large computing power based multi-architecture compatible applications.We systematically introduce the cross-cutting researches of intelligent perception,cognition,and prediction for marine phenomena/processes based on multimodal data technology.First,we clarify the research objects,scientific problems,and typical application scenarios of marine multimodal intelligent computing through the evolution analysis of the lifecycle of marine science big data.Next,we target the differences between ocean data distribution and calculation patterns.We illustrate the uniqueness and scientific challenges of multimodal big marine data on the basis of modeling description,cross-modal correlation,inference prediction,and high-performance computing.1) To bridge the"task gap "between big data and specific tasks for modeling description,we focus on effective feature extraction for related tasks of causality,differentiation,significance and robustness.The ocean-oriented differences and challenges are mainly discussed from six aspects including dynamic changes of physical structure,complex environmental noise,large intra-class differences,lack of reliable labels,unbalanced samples,and less public datasets.2) To construct multi-circle layer,multi-scale and multi-perspective heterogeneous data,the cross-modal correlation modeling is obtained for reasonable integration of multi-model,effective reasoning of cross-model,and the multi-modalities of" heterogeneous gap bridging "through task matching,semantic consistency,and spatio-temporal correlation.The ocean field issue is mainly affected by four aspects of uneven data,large scale span,strong constraints of temporal and spatial,and high correlation of dimensions.3) To fill the" unknown gap "of spatio-temporal information loss in the evolution of ocean,the reasoning and prediction requires the prior knowledge,experience,and reasoning ability in the field of modeling.The main differences of ocean fields are reflected in the three issues of dynamic evolution,spatio-temporal heterogeneity,and non-independent samples.4) To reduce the" computing gap"between complex computing and real-time online analysis of marine super-giant systems,it is necessary to deal with the huge amount of data challenges in high-performance computing problems like increased resolution and the ocean processes refinement of online response analysis.In addition,we sort out and introduce existing work of typical application scenarios,such as marine multimedia content analysis,visual analysis,big data prediction,and high-performance computing.1) Multimedia content analysis:we compare the technical features of existing marine research methods on the five aspects of target recognition,target re-identification,target retrieval,phenomenon/process recognition,and open datasets.2) Visual analysis of marine big data:we summarize the matching issues of dynamic changes of physical structure,high correlation dimensions,and large-scale spans from the perspective of visualization,visualization analysis,and visualization system.3) Ocean multimodal big data reasoning prediction:we review the existing research work from the perspectives of data-driven prediction and prediction of marine environmental variables,construction of marine knowledge graph,and knowledge reasoning.4) High-performance computing issues of ocean multi-modal big data:we introduce and compare the relevant work on the three perspectives of memory-computing collaboration,multi-model acceleration,and giant system evaluation.Finally,we predict the ocean multimodal intelligent computing issues to be resolved and the future direction of it.