A little video of my kitchen that I then camera tracked. This is the process of designating points in the video that get tracked from frame to frame. I chose some of the dots on the counter, a corner of my coffee machine, the top of the door handle, the light on the kettle and you get the idea.
When I set up ten points that were fully tracked from the start of the video to the end, i.e. they smoothly followed the point I placed them on from the start of the video to the end, the fun can start. Using these ten points I can designate a minimum of three of the points that are on the kitchen surface and then three on the back wall. Using these Blender then has the ability to attempt to calculate where the camera was in 3D space, this only took around 3 seconds for this whole video and the result had a very low solve error which is ideal. What this means is that the software has managed to take the 2D points that I tracked in the video and based on knowing which three where the horizon and which three were the wall, it can take that info and the other points and 'solve' where the camera moved in real life. This translates the 2D points into 3D points.
With the camera solved you can assign the virtual camera in Blender to a camera solver constraint. This basically means that the 3D location of the camera that was calculated can be used to move a virtual camera around the 3D workspace in Blender to interact with a modelled scene. This allowed me to import my tree models and gate from a past project and some quickly made grass islands and set them up in the 3D space. So now the camera can move around the models in realtime in the same manner I did with my phone camera in the original video. I can then render the the model with a transparent background as it would appear in real life for each frame of the video and overlay it onto the orignal video which is added as a 2D background.
The one thing I notice after the video had finished rendering (around 12 hours of render time) is that I had not added a plane in the model for the work surface that did not show up in the render. This may not seem a big deal but although the plane would not show up in the render it would capture the shadows of the models making them seem more 'in' the video. Not a big thing but it really would have added a lot.