Neural network under microscope