In a time when Artificial Intelligence (AI) technologies are nearly ubiquitous, railway construction and maintenance systems have not fully grasped the capabilities of such technologies. Traditional railway inspection methods rely on inspection from experienced workers, making such tasks costly from both, the monetary and the time perspective. From an overview of the state-of-the-art research in this area regarding AI-based systems, we observed that their main focus was solely on detection accuracy of different railway components. However, if we consider the critical importance of railway fastening in the overall safety of the railway, there is a need for a thorough analysis of these AI-based methodologies, to define their uncertainty also from a metrological perspective. In this article we address this issue, proposing an image-based system that detects the rotational displacement of the fastened railway clips. Furthermore, we provide an uncertainty analysis of the measurement system, where the resulting uncertainty is of 0.42°, within the 3° error margin defined by the clip manufacturer.