Unmanned aerial vehicles (UAVs) are one of the most promising technologies for weed detection and management because they are not constrained by ground conditions, can be easily maneuvered, and can cover a large agricultural area. For effective use in a UAV-based system, the weed detection system must meet stringent requirements regarding power consumption, overall detection performance, real-time capability, weight, and size. We evaluate the three lightweight deep learning architectures, YOLOv4-tiny, YOLOv4-tiny-3l, and YOLOv7-tiny, for real-time weed detection in horticulture, specifically forest tree nurseries. The deep learning models are evaluated on the three edge computing devices, NVIDIA Jetson Nano, NVIDIA Jetson TX2, and NVIDIA Jetson Xavier NX, that fit the requirements for UAV-based applications. The deep learning models are trained and evaluated with a custom dataset of 921 images and 52,081 individual weed and horticultural plants under normal commercial growing conditions. The deep learning architecture YOLOv4-tiny-3l achieves the highest F₁-score with 80.56% and has the highest average precision for weeds and three of the five horticultures. Moreover, an mAP@0.5 of 83.38% and mAP@0.5:0.95 of 46.89% were attained. The NVIDIA Jetson Xavier NX achieves the highest detection speeds for all image input sizes, with over 60 FPS at FP16 and over 100 FPS at INT8 for an image input size of 832 × 832 pixels, making it suitable for use in UAV-based real-time weed detection in horticulture.