As the variety of products and manufacturing processes increases, the expansion of flexible training approaches is crucial to support the development of human skills. This study presents a model for skill transfer support that extracts experts’ relevant skills as actions and objects relevant to the action into a computational model for transferring skills. This model engages two modes of deep learning as the groundwork, namely, convolutional neural network (CNN) for action recognition and faster region-based convolutional neural network (R-CNN) for object detection. To evaluate the performance of the proposed model, a case study of the final assembly of a GPU card is conducted. The accuracy of CNN and faster R-CNN are 95.4% and 96.8%, respectively. The goal of this model is to guide junior operators during the assembly by providing step-by-step instructions in performing complex tasks. The present study facilitates flexible training in terms of adapting new skills from skilled operators to naïve operators by deep learning.