Maybe you are gripping the bat too closely. It is a known fact that you can swing the bat more efficiently and quicker by not gripping it too tightly. It's the exact same exact principle for a pitcher gripping the ball tighter to slow down the speed of the pitch to fool the batter. Gripping the bat too closely will actually slow down your bat speed like it slows down a pitch from the pitcher.