Surveillance video from the store showed the mother was only out of the car for a few minutes and stayed within view of the car.
"The car and the baby were returned to the distraught mother," police said.
Police added that the baby is doing just fine.
No arrests were made.
The incident is still under investigation.