Amicalola Falls State Park is where the famous Appalachian Trail starts, which goes all the way up to Maine.
Except for that, the state park also features a pretty high waterfall, Amicalola Falls. In spring, it'll probably carry more water then in fall, when we were there. But it's a nice, short trail to the falls, especially in fall, when the trees get all kinds of colors. Weather permitting, you might also see the skyline of Atlanta in the distance.