The realization of automatic identification and positioning of tea buds is the basis for the development of high-quality tea intelligent picking equipment. Aiming at the problems of tiny tea buds and the picking environment which is greatly affected by light, this research proposes a tea bud recognition method based on a deep learning network model to carry out the light source design of the recognition system, which can provide technical support for the realization of all-weather and high-efficiency intelligent tea bud picking equipment. First, an aluminum alloy frame was constructed to provide a closed and shaded dark environment; then, three combinations of heights and three combinations of light intensities were created by adjusting the height of the crossbar and the brightness of the light source; finally, the image datasets of tea buds in different combinations were collected, and recognition tests were carried out on one bud and one leaf and one bud and two leaves by using the improved YOLOv5 model. The experimental results show that the overall accuracy of YOLOv5s is 77.13%, and the overall average precision mean is 86.14%, the overall accuracy of the improved recognition model YOLOv5s-SPD is 80.30%, and the overall average precision mean is 87.3%, and the average detection time of a single image is 5.7 ms, which meets the requirement of real-time detection, and is better than the original YOLOv5s with an overall accuracy improvement of 3.17% and an overall average precision mean of 1.16%, which effectively improves the recognition performance of tea buds. Under the condition of height 90 cm and luminance L7(0.164~0.328 ), the detection accuracy, recall and AP average of one bud and two leaves were 86.70%, 92.45% and 95.00%, respectively. The method proposed in this paper can effectively and quickly detect tea buds, and the light source design scheme supports the development of all-weather high-quality tea intelligent picking equipment.