When it comes to speed, what I’ve noticed is that it takes about 2-3 hours of listening to a new medium to adapt. So keep trying for a while, and it should become a bit easier. If after 3 hours you still can’t understand any of the audio after the second listen, then I would keep working at elementary/preintermediate for a while. Definitely if speed keeps being a problem, I’d try to learn how to use the vocabulary actively, and try to avoid translation because that will slow you down.
Also keep in mind that the “recommended number of lessons” is hopelessly optimistic. For a native English speaker, it’s supposed to take about 2,000 classroom hours to reach fluency in Mandarin (according to government agencies), but the recommended number of lessons would only last 150 hours, make it 300 hours if you include review and you’ll still be a long ways away. So it’s not just you: language learning takes time.