The Fort Monroe Visitor Center (link) at 20 Ingalls Road in Hampton, Virginia, has an excellent exhibit on the first former slaves to escape Confederate lines to freedom.

From the interpretive markers around the statue- On May 23, 1861, three enslaved men Frank Baker, James Townsend, and Shepard Mallory escaped from Confederate lines at Sewell’s Point and rowed across Hampton Roads under the cover of darkness to Fort Monroe. Having lived a life of enslavement, these three men made the bold decision to risk everything for a new life. In this way they created their own legacy of freedom. Thousands of other African Americans followed. This sculpture is dedicated to the freedom seekers who sought refuge at Fort Monroe.

Other pictures from the exhibit.

Further information from the Battle of Big Bethel post.

Benjamin Butler arrived on May 18th to take command of Fort Monroe and five days later three slaves escaped across rebel lines to Fort Monroe (Shepard Mallory, Frank Baker and James Townsend). The following day a Confederate Major, John Baytop Cary, under a flag of truce, requested their return as per the fugitive slave law. Cary stated ” I am informed that three Negroes belonging to Colonel Mallory have escaped within your lines. I am Colonel Mallory’s agent and have charge of his property. What do you mean to do with those Negroes?” Butler replied, “I intend to hold them.” “Do you mean then to set aside your constitutional obligation to return them?” Cary asked. To which Butler responded “I mean to take Virginia at her word. I am under no constitutional obligation to a foreign country, which Virginia now claims to be.” “But you say we cannot secede and so you cannot consistently detain the Negroes.” Cary said. Butler retorted “But you say you have seceded, so you cannot consistently claim them. I shall hold these Negroes as contraband of war, since they are engaged in the construction of your battery and are claimed as your property.” Butler’s “contraband of war” argument justified keeping slaves that had crossed to freedom while simultaneously injuring the Confederate war effort. Word of mouth traveled quickly and within a month over 500 addition slaves escaped to the protection of Fort Monroe. Passage of the Confiscation Act in June of 1861 legalized the process.