Planned in 1803 on the recommendation of Governor King in Sydney, the Sorrento settlement was intended to provide a British base against possible French intrusion into the recently discovered Bass Strait, a haven for sealers fearing bad weather and a secondary settlement for the increasing number of convicts, especially Irish rebels, arriving in Sydney.
Colonel David Collins, formerly judge-advocate of New South Wales, commanded the expedition, which carried some 300 male convicts, a guard of marines and a handful of free settlers, as well as 40 women and 38 children. It reached Port Phillip in early October and landed at Sullivan's Bay, near modern Sorrento.
Collins soon found this site unsatisfactory. The difficult entrance to Port Phillip made it unsuitable as a future commercial port, as a refuge for fishing vessels or as a defence base. Poor soil and timber and lack of fresh water created difficulties for a penal settlement. The party Collins sent to explore Port Phillip did not reach the good land around the Yarra and Maribyrnong rivers, and its conflict with the Watha wurrung people on Corio Bay provided an excuse for him to say that local Koories would be troublesome.
Collins recommended to King that he move to the Derwent River in Van Diemen's Land, and receiving permission he left in two shiploads on 30 January and 20 May 1804. Huts and gardens were abandoned and few relics remain. The four unnamed graves preserved in the area have been found to be those of later pioneers.
One man, William Buckley, a convict who had absconded, was left behind but managed to survive by living with Aboriginal people near Geelong. He joined Batman's party at Indented Head in July 1835.