I am often invited to review papers on deep learning for medical images. Unfortunately many papers do the same mistake; they split data into training/validation/test on the slice/image/patch level instead of on the patient level. This will lead to inflated test scores, as images from the same patient then can appear in both training and test sets. Since these images can be very similar, the network will perform extremely well on the test images. If you use 2D networks on 3D data, the 2D slices have to be split on the patient level, not on slice level. If you use 2D networks on a dataset where each patient has several 2D images, the images have to be split on the patient level, not on image level. If you use deep learning for digital pathology, the patches have to be split on the patient level, not on patch level. nature.com/articles/s41598-0… link.springer.com/chapter/10… nature.com/articles/s41597-0…