Saint Peter's on the corner of Second and C streets S.E. was the second parish church in the District. Daniel Carroll of Duddington had given a tract of land where church sits, to his brother the Bishop. A Nicholas Young contributed an entire square for a cemetery. The corner stone was laid bv Archbishop Mareschal in 1817 and the first Mass was said on October 4, 1821. The first pastor was Rev. James Lucas of Cavan Ireland.
The Rev. Jeremiah O'Sullivan also served as rector of the church in about the 1870s.
O'Sullivan was born in Kanturk, County Cork, to John and Mary (née Ahern) O'Sullivan.
He arrived in the US in 1863, and entered St. Charles College in Ellicott City, Maryland and finished his theological studies at St. Mary's Seminary in Baltimore. He was ordained to the priesthood by Archbishop Martin John Spalding on June 30, 1868.
His first assignment was as a curate t St. Peter's Church in Rockville and was later moved to Westernport, Maryland for nine years and was then sent to f St. Peter's where he served as pastor. On June 16, 1885, O'Sullivan was appointed the fourth Bishop of Mobile, Alabama, by Pope Leo XIII. O'Sullivan died at age 54, and is buried in the crypt of Immaculate Conception Cathedral.