Though public opinion research owes a strong debt to commercial marketing techniques, the field remains deeply connected to the settlement house movement. The movement was a late 19th-century effort to alleviate poverty through the establishment of community centers in poor urban areas. Live-in middle-class reformists provided education, health care, and other services to residents. These social workers turned to surveys and statistical data mining to alleviate the challenges of industrial capitalism.
Social survey research informed the sense of reform politics that percolated throughout the college settlement house movement. The social work pioneers hoped that data would reveal the true experience of the poor and in so doing would point the broader public to the strain and challenges imposed by the industrial world. From these early survey efforts, we have some of our richest statistical pictures of the black community in the late 19th and early 20th centuries. Du Bois’s seminal portrait of an urban black community, The Philadelphia Negro, emerged from this web.
Until Du Bois arrived in Philadelphia, black Americans had limited relationships with the settlement house movement. Though black Philadelphians suffered under high unemployment and poverty rates, the settlement house movement had long excluded blacks from many of their educational and social service initiatives. However, Susan Wharton, a prominent member of the college settlement movement, carved out a role as an advocate for the black community in Philadelphia.
After graduating from Vassar in 1884, Wharton had already developed a long track record of social service for the black community. In 1884 she opened a public library for black children and later in 1888 founded a local branch of the Octavia Hill Association which provided affordable homes for the poor in Philadelphia. In 1895, she wrote to Charles Harrison, her then-neighbor, to see if the University of Pennsylvania might help develop “a plan for the better understanding of the colored people, especially of their position in this city.” It is Wharton who is credited with asking Harrison to extend an invitation to Du Bois to conduct survey research in the black community of Philadelphia.
According to historians Michael Katz and Thomas Sugrue, Du Bois was skeptical about the project. Because of the settlement house movement’s complicated relationship with urban black communities, he suspected that the prime motivation for studying Philadelphia’s black community was political. Indeed, the black community in Philadelphia’s Seventh Ward had been single-handedly thwarting the attempts by several settlement house workers to elect one of their compatriots as ward school director. The settlement house advocates hoped with a new ward school director they might root out some of the rampant corruption in local government politics. But, the Seventh Ward community remained steadfast supporters of the Republican Party machine despite the charges of corruption. Du Bois understood this, even if his colleagues did not: the ward politics of the Republican Party ensured patronage jobs for Black residents. Du Bois suspected the proposed project was intended to help sway voters.
Nonetheless, Du Bois accepted Harrison’s invitation and arrived at the Philadelphia Settlement house in 1896 with his wife, Nina Gomer. As social survey research was in its fledgling stages, Du Bois did not begin the project with a set of research methods. Rather, he set out for himself the enormous task of personally interviewing 5,000 black Philadelphians, attempting to achieve full coverage of residents within a carefully defined geographical frame. He focused his attention on typical subjects of interest in 19th-century social science and social work: family and criminality. Yet, Katz and Sugrue emphasize that Du Bois maintained a historical perspective to his understanding of social development. To understand the experience of black Americans was necessary to contend with the vicissitudes of history. He set out to explore pathology and deviance, but rather social development and historical change.
Map of an interview area from “The Philadelphia Negro,” W.E.B. Du Bois, 1899, University of Pennsylvania Press, image from the Internet Archive https://archive.org/details/philadelphianegr00dubo/page/46/mode/2up
Du Bois was not often received well in the homes of interviewees. He mostly recorded his interviews on paper but, perhaps because of the challenge in striking up a rapport with his interviewees, utilized “information [he] stored in [his] memory and wrote out as memoranda,” for the final manuscript. His research surprised him. Du Bois later noted, “I concluded that I did not know so much as I might about my own people…I had learned far more from Philadelphia Negroes than I had taught them concerning the Negro Problem.”
Though the American Sociological Review, reflecting their own racial bias, refused to review the study’s final publication, other journals and scholars gave Du Bois’s work high praise. The Yale Review enthusiastically declared that the Philadelphia Negro was “not merely a credit to its author and to the race of which he is a member; it is a credit to American scholarship and a distinct and valuable addition the world’s stock of knowledge concerning an important and obscure theme.” Indeed, Du Bois’s work and much of the work from the settlement house movement, including Jane Addams and Florence Kelley’s Hull-House Maps and Papers (1895), are the progenitors of today’s public opinion research. Du Bois later reflected after the publication of The Philadephia Negro that “the social problem is ever a relation between conditions and action and…conditions and actions vary and change from group to group, from time to time, and from place to place, social problems change, develop and grow.” Public opinion research, building on this rich lineage of sociological study, provides that same window to social change over time.
-by Kwelina Thompson
2020 Roper Center Kohut Fellow
To learn more about the life of W.E.B. Du Bois, visit the Roper Center's biography in the Pioneers of Polling section of this site.
 Katz, Michael B., and Thomas J. Sugrue, eds. "W.E.B. Du Bois, Race, and the City: The Philadelphia Negro and It's Legacy." University of Pennsylvania Press, 1998.
Du Bois, W.E.B., “The Philadelphia Negro," University of Pennsylvania Press,1899.