Automated monitoring of pigs for timely detection of changes in behavior and the onset of tail biting might enable farmers to take immediate management actions, and thus decrease health and welfare issues on-farm. Our goal was to develop computer vision-based methods to detect tail biting in pigs using a convolutional neural network (CNN) to extract spatial information, combined with secondary networks accounting for temporal information. Two secondary frameworks were utilized, being a long short-term memory (LSTM) network applied to sequences of image features (CNN-LSTM), and a CNN applied to image representations of sequences (CNN-CNN). To achieve our goal, this study aimed to answer the following questions: (a) Can the methods detect tail biting from video recordings of entire pens? (b) Can we utilize principal component analyses (PCA) to reduce the dimensionality of the feature vector and only use relevant principal components (PC)? (c) Is there potential to increase performance in optimizing the threshold for class separation of the predicted probabilities of the outcome? (d) What is the performance of the methods with respect to each other? The study utilized one-hour video recordings of 10 pens with pigs prior to weaning, containing a total of 208 tail-biting events of varying lengths. The pre-trained VGG-16 was used to extract spatial features from the data, which were subsequently pre-processed and divided into train/test sets before input to the LSTM/CNN. The performance of the methods regarding data pre-processing and model building was systematically compared using cross-validation. Final models were run with optimal settings and evaluated on an independent test-set. The proposed methods detected tail biting with a major-mean accuracy (MMA) of 71.3 and 64.7% for the CNN-LSTM and the CNN-CNN network, respectively. Applying PCA and using a limited number of PCs significantly increased the performance of both methods, while optimizing the threshold for class separation did result in a consistent but not significant increase of the performance. Both methods can detect tail biting from video data, but the CNN-LSTM was superior in generalizing when evaluated on new data, i.e., data not used for training the models, compared to the CNN-CNN method.