This is my favorite part!
But yeah, that's about right. There are so many tiny details that it's hard respond to. For example the ideal impulse response is not where some frequencies end before other frequencies. The ideal frequency response would be the cone instantaneously moving to maximum full-forward travel and then instantly moving back to rest position, so fast you would barely notice it. On a measurement system the ideal impulse response would be a single digital 1 surrounded by a bunch of zeros before and after, kind of like below.