Uncovering the heterogeneity of cellular populations and multicellular constructs is a long-standing goal in fields ranging from antimicrobial resistance to cancer research. Emerging technology platforms such as droplet microfluidics hold the promise to decipher such heterogeneities at ultra-high-throughput. However, there is a lack of methods able to rapidly identify and isolate single cells or 3D cell cultures. Here we demonstrate that deep neural networks can accurately classify single droplet images in real-time based on the presence and number of micro-objects including single mammalian cells and multicellular spheroids. This approach also enables the identification of specific objects within mixtures of objects of different types and sizes. The training sets for the neural networks consisted of a few hundred images manually picked and augmented to up to thousands of images per training class. Training required less than 10 minutes using a single GPU, and yielded accuracies of over 90% for single mammalian cell identification. Crucially, the same model could be used to classify different types of objects such as polystyrene spheres, polyacrylamide beads and MCF-7 cells. We applied the developed method for the selection of 3D cell cultures generated with Hek293FT cells encapsulated in agarose gel beads, highlighting the potential of the technology for the selection of objects with a high diversity of visual appearances. The real-time sorting of single droplets was in-line with droplet generation and occurred at rates up to 40 per second independently of image size up to 480 × 480 pixels. The presented microfluidic device also enabled storage of sorted droplets to allow for downstream analyses.