The Old Fort prison was later extended to include "native" cells, called Section 4 and Section 5, and, in 1907, a women's section was added, the Women's Gaol. An awaiting-trial block was constructed in the 1920s. Both political activists opposed to apartheid and common criminals were held at the prison, and striking white mineworkers in 1907, 1913 and 1922. During the apartheid era the prison complex became a detention centre for political dissidents opposed to apartheid, striking white mineworkers (in 1907, 1913 and 1922), those deemed "anti-establishment" and those who simply violated the pass laws of the time. Mahatma Gandhi was imprisoned here in 1906. Nelson Mandela, Joe Slovo, Bram Fischer, Albert Lutuli and Robert Sobukwe are some of its famous prisoners. For this reason it was also called ''The Robben Island of Johannesburg'' due to how infamous it became as a result of holding these political prisoners. The site housed prisoners until 1983, when it was closed.
Activists were usually held as awaiting trial prisoners and then sent off to Robben Island or Pretoria to serve jail terms. Brewing beer – an illegal activity if you were black – also landed many women in jail, a situation depicted in the Women Gaol. Still others were arrested for having sex across the colour bar or for homosexual sex. Amongst the practices reported during detention, the humiliating "Tauza" dance, the beatings in the notorious Number Four prison for black men, the detention for months in dirty, overcrowded conditions in the Awaiting Trial Block, or being stripped of their underclothes and their dignity in the Women's Gaol.
The Old Fort was declared a National Monument in 1964 although it continued as a functioning prison until 1987, after which the buildings and the site as a whole suffered from neglect and vandalism. Constitution Hill opened its doors as a museum in 2004, with tours taking the visitors to three prison museums: Number Four, the Women's Gaol and the Old Fort. The site also features the Constitutional Court and the court's collection of 200 South African artworks.