目的 由于光在水中的衰减/散射以及微生物对光的吸收/反射等影响，水下图像通常存在色偏、模糊、光照不均匀以及对比度过低等诸多质量问题。研究人员对此提出了许多不同的水下图像增强算法。为了探究目前已有的水下图像增强算法的性能和图像质量客观评价方法是否适用于评估水下图像，本文开展大规模主观实验来对比不同水下图像增强算法在真实水下图像数据集上的性能，并对现有图像质量评价方法用于评估水下图像的准确性进行测试。方法 构建了一个真实的水下图像数据集，其中包含100幅原始水下图像以及对应的1 000幅由10种主流水下图像增强方法增强后的图像。基于成对比较的策略开展水下图像主观质量评价，进一步对主观评价得到的结果进行分析，包括一致性分析、收敛性分析以及显著性检验。最后将10种现有主流的无参考图像质量评价在本文数据集上进行测试，检验其在真实水下图像数据集上的评价性能。结果 一致性分析中，该数据集包含的主观评分有较高的肯德尔一致性系数，其值为0.41；收敛性分析中，所收集的投票数量与图像数量足够得到稳定的主观评分；表明本文构建的数据集具有良好的有效性与可靠性。此外，目前对比自然图像的无参考图像质量评价方法并不适用于水下图像数据集，验证了水下图像与自然图像的巨大差异。结论 本文构建的真实水下图像数据集为未来水下图像质量客观评价方法以及水下图像增强算法的研究提供了参考与支持。所涉及的图像以及所有收集的用户数据，都在项目主页(https://github.com/yia-yuese/RealUWIQ-dataset)上公开。

Objective Underwater images processing are essential to marine issues in the context of defense, environmental protection and engineering. However, there are always severe quality degradation issues like color cast, blur, and low contrast are greatly restricted the quality of underwater imaging and operation systems due to the inner-water light attenuation/scattering and the microbe derived absorption/reflection of light. Underwater image enhancement (UIE) algorithms have been demonstrated to improve the quality of underwater images nowadays. The two aspects of challenges are critical to be illustrated as below:one of huge gap between the synthesized and in-situ underwater images processing are constrained of complicated degradations of diverse underwater environments. The other is challenged that the existing objective image quality metrics are matched to evaluate the in-situ quality of various UIE algorithms. To resolve the above two issues, our demonstration has illustrated as following:first, we build up a real-world underwater image quality evaluation dataset to compare the performance of different UIE algorithms based on a collection of in-situ underwater images. Next, we evaluate the performance of existing image quality evaluation metrics on our generated dataset. Method First, we collect 100 real-world underwater images, including 60 color cast-dominant and 40 blur-dominant ones, and apply 10 representative UIE algorithms to enhance the 100 raw underwater images. A total number of 1 000 enhanced results (10 results for each raw underwater image) are generated. Next, we conduct complex human subjective studies to evaluate the performance of different UIE algorithms based on the pairwise comparison (PC) strategy. Thirdly, we analyze the results obtained from our subjective studies to demonstrate the reliability of our human subjective studies and get insights on the pros and cons of each UIE algorithm. The Bradley-Terry (B-T) model on the PC results obtained B-T scores as the ground truth quality scores of the enhanced underwater images. Finally, we test the capabilities of existing image quality metrics via the correlations between the B-T scores and the predicted 10 existing representative no-reference image quality metrics for evaluating UIE results. Result We illustrates the Kendall coefficient of inner subjects' protocols, a convergence analysis and conducts a significance test to verify the dataset. The Kendall coefficient of inner subjects' protocols on the full-set is around 0.41, which demonstrates a qualified inter-subject consistency level. Such coefficient is slightly different on the two subsets, i.e., 0.39 and 0.44 on the color cast subset and blur subset, respectively. In respect of the convergence analysis, the mean and the variance of each underwater image enhancement algorithms tend to be clarified in the context of the increasing of the number of votes and the number of images.The similar subjective scores are obtained for each underwater enhancement algorithms. The significance for test results demonstrates that GL-Net is the best and underwater image enhancement convolutional neural network(UWCNN) is the worst for the adopted 10 UIE algorithms. In addition, there is slight difference on the performance rankings of different UIE algorithms on the two subsets. Finally, an existing no-reference image quality metrics can be unqualified for UIE algorithms evaluation. Conclusion Our first contribution is based on an in-situ underwater image quality evaluation dataset through conducting human subjective studies to compare the performance of various UIE algorithms with a collection of in-situ underwater images. The other one is that the performance of existing image quality evaluation methods is evaluated based on our dataset and the limitation of the existing image quality metrics is identified for UIE quality evaluation. Overall, this research targets underwater image quality evaluation metrics. All the images and collected data involved are available at:https://github.com/yia-yuese/RealUWIQ-dataset.