Most Ohio state parks let you take nothing home. Hueston Woods is the exception: its naturalist says the creeks are so full of Ordovician fossils that collecting for your own collection is allowed. The richest ground is the stream cuts at the north end of Acton Lake and the spillway at the south end, where thin limestone and shale let brachiopods, bryozoans, horn corals and the odd trilobite weather loose. Keep it personal, never commercial, and ask at the nature centre which ground is open, since the park’s nature preserve is off limits.
Brachiopods, horn corals, crinoids, trilobites · Late Ordovician, ~450 mya
Open to the public for collecting, and what you find is yours to keep. Check the posted rules before you start: they can change with the season and the landowner.
Other places on the map within a couple of hours.