The gorilla, Harambe, possibly agitated by the screams of onlookers, at first didn't seem to hurt the boy, witnesses told CNN.
But when onlookers started to scream, it appeared that the gorilla wasn't going to let the child go.
"From what we saw, (the child) could have been killed at any second. (Harambe) threw him 10 feet in the air, and I saw him land on his back. It was a mess," said Bruce Davis, a visitor to the Cincinnati Zoo, CNN reported.
Kimberley Ann Perkins O'Connor told CNN that she heard the boy joking with his mother about going into the moat. The boy was in the water a few minutes later.
Zoo officials decided to shoot, rather than tranquilize, the gorilla because they said the dart could have made the animal angry and might have taken too long to kick in.
An online petition on Change.org is seeking justice for Harambe by asking for criminal charges against the boy's mother. It has more than 334,000 signatures.