Port Phillip was never strictly a penal settlement, but as part of New South Wales it received a number of convicts from time to time. First, convict labourers were sent down, about 100 at a time, to provide government labour between 1836 and 1842, but most of these men were sent back to Sydney after relatively short terms and there were complaints that more were needed. Second, a number of squatters from central New South Wales brought down some of their 'assigned servants' who were convicts. How many is uncertain - there were probably about 550 in the district in 1842 - and many of these remained in Port Phillip when their sentences expired. Third, between 1844 and 1849, 1727 'exiles' known as Pentonvillians were sent out with tickets of leave.
However, most important were time-expired prisoners from Van Diemen's Land and New South Wales. Again the number is uncertain. No records were kept of the latter, as they were moving within the same colony, though probably they were fewer than those from Van Diemen's Land. These are thought to have numbered over 3000 in 1846 (about 15% of the male population). Probably another 2000 had arrived by 1848, and after a temporary decline, many more came after the gold discoveries in 1851.
Though all too often described as scoundrels, villains and ruffians, these men provided an important supply of labour to squatters in the 1840s. But they also committed 40% of the district's crime between 1841 and 1846 (and more later) and their presence produced strong criticism of the imperial authorities for trying to block the colony's attempts to stop their arrival.