The insider also reported that a firm deadline has been set for Ferguson and ex-Prince Andrew to leave their longtime home at Royal Lodge.