One hundred babies spent time living in UK prisons last year.
The figures have prompted Prime Minister David Cameron to demand a review of how to deal with new mothers in custody.
Cameron who will give a speech on prisons policy on Monday, confirmed the government is looking at alternatives to jail, Sky News reported
He said: “It is absolutely terrible to think that some babies are spending the earliest months - even years - of their lives behind bars.
“Prison staff does their best to make these environments pleasant. Some units even have special sensory rooms, so that babies can see colors, sights and sound - even nature - that they wouldn’t ever otherwise see inside the grey walls of a jail.
“There are actually women in these prisons who were born in the same prison 20 years earlier, and then have ended up there later as criminals themselves.
“Think of the damage done to the life chances of these children.”
Pregnant women who give birth while serving a prison sentence are usually transferred to a jail’s mother and baby unit for an initial period.
These units have to be able to accommodate babies up until the age of 18 months.
Cameron said: “It’s time to think seriously about whether this is the right approach. We’ve got to break this cycle.
“So I want us to find alternative ways of dealing with women prisoners with babies, including tagging, problem-solving courts and alternative resettlement units.”
He also confirmed a pilot scheme to track offenders by satellite will be launched this year. The technology will be introduced nationwide by 2020.