Anthony Albanese made history on Saturday by becoming the first prime minister to get married while in office.