AR relies on two functions: GPS location and compass. Please ensure these are turned on and the app has permission to use them.
GPS location can take a few minutes to be accurate, especially if you have not used it in some time. Waiting with the app active will slowly give increased accuracy. If a GPS signal is not available, such as inside a building, AR may incorrectly estimate your location.
Your built-in compass needs regular calibrating in order to be accurate. In iOS Use the built-in compass calibration system by turning it on at Settings > Privacy >Location Services > System Services > Compass Calibration and then use the built-in compass app to trigger calibration.
In Android, you can download a compass calibration app, or with the AR mode open just wave the device around in a figure-eight for around 30 seconds to improve accuracy.
If the compass is inaccurate, features will not line up in AR view.