I believe that the low flight rate currently projected for SLS and Orion is a cause for concern. No human-rated launch system in NASA's history has flown so infrequently. With such a low launch rate it would not just be difficult to maintain program momentum; it would be difficult to keep flight teams sharp and mission-ready.
A more serious concern is that the SLS/Orion combination alone is insufficient to carry out missions to any important destinations beyond low Earth orbit. The Orion capsule can support a crew of four for three weeks, which is far too short a time to conduct a mission to an asteroid.
An asteroid mission therefore requires development of another major piece of hardware, capable of providing crew support in deep space for many months. There is no funding in NASA's budget to develop this hardware.
Three weeks is enough time for a mission to the surface of the Moon, which like an asteroid mission could be a reasonable stepping-stone to Mars. But such a mission would require a lunar lander, which again is not in NASA's budget.
So if we truly intend to have a program of human exploration to some destination beyond low Earth orbit, there is a piece of the puzzle missing. SLS and Orion will be highly capable vehicles, and their development is progressing well.
But they are only part of the picture. Without some means to develop or acquire the missing piece — either a deepspace habitation module or a lunar lander — a decade from now NASA will be unable to do much more in deep space than duplicate the success of Apollo 8's historic mission to orbit the Moon, more than half a century later.