1 Introduction

This article explores the relative participation levels of ‘Not In My Backyard’ (NIMBY) proponents vs other voices at public hearings in San Francisco, a city with an exceptionally dire housing crisis. Our hypothesis is that NIMBYism is a key factor that undermines social mix in San Francisco. The NIMBY phenomenon has been discussed in planning literature for decades (see, among others, Dear, 1992; Pendall, 1999; Schively, 2007; DeVerteuil, 2013; Payton Scally & Tighe, 2015). However, it is worth revisiting because it is particularly insidious at a time and place where a mass of poor people are forced to live on the streets and in their cars because they cannot afford any housing, even in the form of a single room. We aim to find out whether NIMBY attitudes dominate planning hearings on housing development proposals, or whether other movements have gained momentum.

Once very diverse, radical, and bohemian, San Francisco has come to have the most expensive housing stock of any US city. It now caters just to a wealthy minority—heavily connected to the tech industries of the neighboring Silicon Valley (Schafran, 2013). As one commentator notes:

technology firms are at the heart of San Francisco’s new political economy but their vision of the city is all about private property and profitability and thus retains a central role for real estate (Stein, 2019:39).

Rents and house prices are no longer affordable to service workers and to the creative and alternative life-style communities, dissidents, and migrants—for which San Francisco was once a special haven (Metcalf, 2015; see also Wehrhahn, 2015; Mayer et al. 2012). Another commentator observes:

The quirky, counter-cultural San Francisco so many of us fell in love with is almost gone now, destroyed by high housing costs. We’ve lost not only the politics, but all kinds of cultural experimentation that just doesn’t thrive in places that are expensive. (Metcalf, 2015:1)

In the decades since the 1960s, there has been a growing chasm between housing costs in San Francisco and the rest of the United States (Fig. 1). As the graphs show, during the current tech boom there has been a particularly intense upward spiral in housing costs. These costs have remained high even during the Covid-19 pandemic. The advent of short-term rental platforms such as Airbnb has had a negative effect too, by removing already scarce housing units from the long-term rental market (BoS 2015). Rental costs are particularly concerning in a city where traditionally, two thirds of residents have been renters (Rosen & Sullivan, 2014).

Fig. 1
figure 1

Trends in rents and house prices (unadjusted), U.S., California, and San Francisco compared. Charts by authors based on data from US Census, Zillow, and California Association of Realtors

Full size image

Under these circumstances, conversations and confrontations between NIMBYs and their opponents take place weekly at the San Francisco Planning Commission Meetings. Which group drives the public discourse at this point? Research has found that in other US cities with a comparable housing crisis, planning meetings are dominated by well-to-do, retired baby boomers (Einstein et al. 2019; Holleran, 2020). Is this the case in San Francisco too? Are older, wealthier, and more powerful homeowners dominating public participation and overshadowing community activism by renters supporting the construction of more housing (Scally & Tighe, 2015)? Or, is the situation here different than elsewhere given the city’s past social mix and current housing stress? Have San Francisco youth embraced anti-NIMBY attitudes? As members of the Y and Z Generations, are they “taking on the boomer-led power structure” (Eskenazi, 2018)?

The dynamics of public meetings in San Francisco may be familiar to planners in other US coastal cities with tight housing markets and high development pressures. They may also be of interest to planners working in less contentious planning environments. Arguably, these public participation processes are what makes planning work both fascinating and challenging. In addition to existing professionals, the article provides some lessons for planning students, in the US and elsewhere, who will soon enter the profession, on what to expect while conducting public meetings. Before proceeding to the empirical study, we succinctly review the concept of NIMBYism and compare it to other movements—the YIMBY movement in particular.

2 Theoretical background on NIMBYism

Coined in the 1970s, the term NIMBY describes active, vocal, and connected residents who protest against new development proposals in their area (in particular mid-rise and high-rise apartment buildings), even where they believe that such developments would be beneficial to a city or region at large (Eranti, 2017; Burningham, 2000; Barlow, 1995). NIMBY parochial interests (and fears) include: preserving class status, excluding lower income households, limiting competition for parking and school placements, preserving views and open spaces, protecting home values, and the like (Wassmer & Wahid, 2019). Restricting housing supply results in shortages which increase the value of existing housing (Gerrard, 1994). Where developments include affordable housing, especially rental housing, a NIMBY tactic is to frame prospective tenants as freeloaders, anti-social, and even potentially criminal (Scally, 2012; Wilton, 2002). As such, the term NIMBY has negative connotations in academic circles (Petrova, 2016).

In the United States, NIMBYism has a strong racial and class dimension as it tends to be perpetuated by the white middle and upper classes, while minority communities are often the victims (Gerrard, 1994; Werner, 1998; Wilton, 2002). Given that these groups hold more socio-economic clout, have stronger political connections, and are more capable of mobilizing the media, they are often (though not always) in a position of taking institutionalized action against new development proposals and pressuring municipalities to adopt various regulatory barriers to new housing (Gerrard, 1994). As Scally (2012:719) explains, these “range from direct exclusion of multifamily development to indirect exclusion through cost inflation caused by imposing growth boundaries, enacting strict environmental controls, requiring low-density development and thwarting infill development, charging excessive fee, and inefficiently moving proposed projects through the permitting pipeline, among other things.” Lawsuits are also common (Werner, 1998).

The NIMBY phenomenon is evident throughout the industrialized world, and does not focus exclusively on new housing development but may target other socially desirable land-uses such as renewable energy plants, assisted living facilities, and even bicycle lanes (Lake, 1993; Hayden Lesbirel, 1998; Galster et al. 2003; England, 2007; Devine-Wright, 2014; Butterworth & Pojani, 2018; Pegler et al. 2020). However, it is especially prevalent in American city-regions, in which power over development permits is decentralized among small jurisdictions and local politicians are accountable to relatively small constituencies (see Dear, 1992; Pendall, 1999; Schively, 2007; DeVerteuil, 2013; Scally & Tighe, 2015).

It is often the case that NIMBY proponents are highly emotional and even irrational or hostile, with limited information on the project to which they are opposed (i.e., the actual risks and benefits) (Doberstein et al. 2016; Gibson, 2005). In reality, the effect of new housing construction on existing home values within a neighborhoods is less than clear (see Wassmer & Wahid, 2019). In some cases, NIMBYism is fueled by nostalgia for a romanticized and now largely bygone era in America when whiteness afforded sacrosanct socio-spatial privileges (Wilton, 2002). There is some evidence that, where new housing construction and residential densification is framed as delivering public benefits in the form of reduced carbon footprint and traffic, local residents are more willing to accept incremental increases. However, this approach is unlikely to overcome NIMBYism (Doberstein et al., 2016).

In some regions in Europe and Australia which are also at risk of housing unaffordability, local governments have moved to reduce the negative impacts of NIMBYism by emphasizing public participation at the policy-setting stage, rather than when an individual development application is submitted. Additionally, local councilors are limited in their involvement with the development approvals process. Many applications, particularly those which have high strategic importance, are at high risk of corruption, and/or are sensitive, are assessed by expert panels (including environmental lawyers and professional planners) rather than politicians (see Baker et al. 2007; PIA 2012; Pacione, 2014).

However, a technocratic approach and a frontloading of public participation activities minimizes opportunities for objections, which also raises concerns. A cornerstone of democratic processes is to allow residents to have a say on a development. It is dangerous to assume that land-use experts and planning agencies are always “morally right” or “rational” – especially in light of the devastating legacy of the modernist movement in cities (Gibson, 2005). As Gibson (2005: 382) observes, “one can easily imagine instances when, by preventing the state from pursuing ill-conceived and costly projects, local opposition movements could be doing the public a big favor.” Local opponents of a development proposal may in fact articulate motives and concerns—for example, environmental protection—that “transcend their own narrow self-interest” (Gibson, 2005:387). In neoliberal planning contexts, where public scrutiny is absent, planning decisions have been known to systematically favor developers (Kwok et al. 2018). In authoritarian contexts, a lack of public participation has enabled governments to impose inappropriate planning decisions on communities that have no recourse (Pojani 2017). The balance between efficiency, fairness, and inclusiveness is a delicate one, and in some cases, NIMBYism may be viewed as a manifestation of collective action and popular resistance (Barlow, 1995; Hager & Haddad, 2015).

Several movements have emerged to counter NIMBYism. One in particular, called ‘yes in my back yard’ (or YIMBYism) will be reviewed in more depth later on as it is a more recent movement and, as evident from the name, has positioned itself in dialectic opposition to NIMBYism. The latter is particularly detrimental in San Francisco, due the critical characteristics of the city’s and region’s land-use, demographic, and economic systems. We summarize these below.

3 Case study context of San Francisco

The City of San Francisco is one of approximately one hundred cities in the metropolitan region (San Francisco Bay Area), with a population of nearly 900,000 out of a metropolitan population of 7 million. San Francisco is surrounded by moderate-sized suburbs. The Pacific Ocean is to the west of the city and there are large bays on the east and north sides. Therefore, there is little room for urban expansion. The median household income in San Francisco is quite high relative to other cities, at over $100,000 per year (US Census, 2020). Alarmingly, a local family is considered as ‘low income’ on a salary of nearly $120,000 per year (Nunn & Shambaugh, 2018). Education levels are also very high with 57 percent of the population possessing an undergraduate or postgraduate degree. While two thirds of the population are employed, the poverty rate is close to the US average, at 11 percent. The city includes many migrants: 43 percent of people speak a language other than English at home, with Spanish and Asian languages dominating. Less than half of the population is White, more than a third is Asian, 15 percent is Hispanic or Latinx, and just 5 percent is Black (US Census, 2020).

The cities that comprise the San Francisco Bay Area have virtually complete powers over new development, and their budgets for municipal services (including schools) are largely locally-based. A major portion of these suburbs are dominated by single-family homeowners. These homeowners feel strong incentives to minimize or preclude additional development, especially of multi-family housing and particularly housing affordable to lower-income groups (Iglesias, 2002; Wheeler, 2001). The foregoing characteristics are typical of the larger metropolitan areas in North America (see Eskenazi, 2018; Doberstein et al. 2016; Been, 2018). However, in the case of San Francisco, the supply system has been especially dysfunctional. While rent levels have soared above the national average since the 1990s (Fig. 1), the rate of new construction of multi-family housing has been low. According to some estimates in the mid-2010s, over the course of several decades, San Francisco should have been building at least 5000 new housing units a year to deal with the growing demand but instead averaged only about 1500 a year (Metcalf, 2015).

In the late 1960s, San Francisco’s progressive culture forged an anti-development stance, which endures into the present (Weiner, 2015). At the time, a preservationist movement was a reaction to urban renewal “bulldozing black neighborhoods, ramming freeways through cities, and foreboding public housing towers” (Metcalf, 2015:1). In that context, the political Left cast itself as a defender of local character against profit-driven developers supported by Governor Ronald Reagan (Beitel, 2012; Rolle & Verge, 2015). But local residents continue to be vocal in their opposition to new development, including housing development, even now that the city faces an extreme affordable housing shortage (Graham, 2018).

Housing demand has skyrocketed in San Francisco over the past few decades (see Fig. 1). A key explanation is a rekindled interest in urban living. Unlike previous generations that sought (and legislated) a conformist, car-oriented suburban lifestyle (Baar, 1992; Hirt, 2015), the Millennial generation seeks “interesting” places and experiences. Rather than moving in pursuit of job opportunities and financial rewards, as it was custom in the past, younger people have become motivated to move to attractive places which provide an urban lifestyle (Florida, 2012; Twenge, 2018). At the same time, opportunities to create new housing by expanding outwards are quite limited in San Francisco by the ocean on one-side and a large bay on two other sides.

In addition to urbanity and topography, in San Francisco housing pressures have been greater than elsewhere due to the waves of “technology gold rush” that the city has experienced (Bowles, 2019; Ferenstein, 2019; Ghaffary, 2019). The ‘dot com booms’ of the late 1990s and 2010s resulted in an influx of lucrative tech companies moving to Silicon Valley and their employees moving to San Francisco. Between 2011 and 2017, the city added 175,000 jobs but built fewer than 20,000 housing units (Graham, 2018). Most new units were in the form of luxury high-rise development targeting upper-income residents rather than serving the city’s poor (Hartman & Carnochan, 2002; Stein, 2019). Hence gentrification and displacement of low socio-economic communities ensued, and San Francisco’s socio-economic mix was undermined (Beitel, 2012). One commentator recounts:

Historically, the south side of the city was racially and ethnically mixed, a patchwork of different communities brought by the work that has pulsed through the city, as well as the home of the nation’s most prominent queer community. But as industrial work trickled out of the city, the tech industry moved in, bringing highly educated white people who make lots of money into formerly working-class areas. (Madrigal, 2019: 5)

In 2019, the median house price was an astonishing US$1.3 million whereas average rents were more than $1700 per month (McCamy, 2019). Residents are now struggling to meet their rent payments; buying a house is no longer an option for most (Mayer et al. 2012). Cohen and Martí (2016:1) of the Council of Community Housing Organizations note that.

more than 4100 housing units have been removed from the affordable housing supply over the last 10 years through condo conversions, demolitions, owner move-in evictions, Ellis Act evictionsFootnote 1 and other means (not including tenant “buyouts” which are dramatically on the rise as well). Across the City there is a ratio of three existing units lost for every four new affordable units produced.

This means that the stock of existing affordable units is being eroded at a faster pace than the speed of constriction of new affordable units. Those who cannot leave to other cities are moving into houseboats or even decrepit and unanchored vessels on the bay to escape the burden of increasing rent prices (Carlton, 2019). The homeless population, already quite high, increased by 17 percent between 2017 and 2019, to reach about 7000 (Ho, 2019). Now the city has one of the highest rates of income inequality in America (Reuell, 2019). Even so, a common tactic employed by NIMBY groups is to pressure local officials to put affordable housing projects through the so-called CEQA process (California’s Environmental Quality Act, which mandates an Environmental Impact Report), thereby considerably slowing or even halting construction (Wassmer & Wahid, 2019).

One California law from 1978, Proposition 13, fueled NIMBYism considerably. It rolled property taxes back to 1976 values, with very limited annual rises (properties could only be reassessed after a sale). Under these circumstances, adding new housing would strain local finances as building costs would be higher than property tax revenues. Given a housing crunch, old-time homeowners became increasingly motivated to embrace NIMBYism than under a regime of rising property taxes. Moreover, in the current context of exorbitant housing prices, new homeowners who have recently stretched to purchase a house are extremely motivated too to protect their investment (Dougherty, 2020).

Under these circumstances, a new movement, YIMBYism, has emerged (McCormick, 2017; Warburg, 2017), which is now spreading beyond the United States to Canada, Australia, and Europe (McCormick, 2017). YIMBY activists turn NIMBY arguments on their head by arguing that new housing development is positive in terms of social mix, as it provides more opportunities for shelter to more people (Rayment, 2016). While not universally applauded, the movement is gaining traction and media attention, and an official group called ‘YIMBY Action’ has been launched (www.yimbyaction.org).

While the term YIMBY was coined in the last few years, the idea of a portion of San Franciscans supporting construction and being pro-development is not new. One of the early documented cases of YIMBYism was a protest during California’s housing crisis of the 1980s. Locals requested planning reforms and relaxation of land-use regulations in order to lower housing costs (Gray, 2018b). In 1991, a special housing commission, which was federally appointed to study local barriers to the construction of affordable housing, produced a widely circulated report, ‘Not in My Back Yard: Removing Barriers to Affordable Housing’ (US Department of HUD 1991). The report questioned the continuing legality of the bedrock of U.S. land-use planning, the single-family only zone.

However, the YIMBY movement has also attracted critics, who denounce it for promulgating market-driven solutions to the San Francisco housing crisis, under a ‘build, baby, build’ mantra. This approach is seen as supporting unrestrained capitalism, anti-regulation, whitewashing, neoliberal multi-culturalism, and racialized gentrification (Dougherty, 2020; McElroy & Szeto, 2017). Because San Francisco lacks physical space for expansion, the development of market-rate condos may imply the demolition of existing houses and businesses and the displacement of lower-income tenants (Rosen & Sullivan, 2014). While an increased supply of market-level units may alleviate housing pressures at the regional level, this effect may not be felt at the block level in San Francisco – certainly not in the short term. Here, the unmet demand for housing is so extreme that new construction invites gentrification and augments racialized surveillance and criminalization of the poor. Filtering—a trickle-down process by which older market-rate units become more affordable as new, shiny units are inserted into the market—may take years to materialize. In much of San Francisco there may be no filtering at all—especially where older units are in heritage buildings (e.g., Victorian or Art Deco houses) which are aesthetically desirable to the educated middle-classes (McElroy & Szeto, 2017; Zuk & Chapple, 2016).

Meanwhile, displaced and dispossessed people (often ethnic minorities) may find it difficult to access affordable housing in Bay Area cities that are located within a reasonable commuting distance of San Francisco because those cities enforce single-family zoning regulations, which keep out the poor. Therefore, they are pushed out to far flung exurbs (Schafran, 2013; Walker & Schafran, 2015). Dougherty (2020:90–97) provides a list of incensed public comments received by a suburban municipality in the San Francisco Bay Area when a new housing development was proposed:

“Traffic,” “too aggressive,” “not respectful,” overburdening,” “an embarrassment,” “outraged,” “audacity,” “very urban,” … “will allow more crime to be committed,” ... “deeply upset,” “unsightly,” “monstrosity,” “inconceivable,” “simply outrageous,” “vehemently opposed,” “sheer scope,” “too much,” “semi-rural,” “very wrong,” “blocking views,” “inconsistent,” does not conform,” … “property values will be destroyed,” … “the project will strain our already underfunded schools,” “…there is already plenty of housing available within commuting distance of San Francisco,” [and] “don’t expect my vote in the future if this passes”

Given extreme decentralization and powerful suburban communities, new market-rate housing development in San Francisco needs to go hand in hand with other interventions including eviction protection, rent stabilization, low-income housing construction, residential hotel construction, inclusionary zoning, relocation assistance, and job-housing linkage programs (Rosen & Sullivan, 2014; Walker, 2018). A number of non-YIMBY groups such as the Tenants Union, Tenants Together, Our Mission No Eviction, Housing Rights Committee, Causa Justa, Council of Community Housing Organizations (Choo Choo), Coalition on Homelessness, Plaza 16 Coalition, and others are pushing for these types of solutions, which help the poor and promote social mix. Often, the members of these organizations are working-class tenants of color. However, they too are fraught with internal conflicts and clashing goals, and often have to make tradeoffs and uneasy alliances with other groups, including NIMBYs (Dougherty, 2020:122).

Obviously, the NIMBY and YIMBY movements do not stand in dialectic opposition. The picture is more much complex. While YIMBY activists propose market-rate housing as an important solution, they do not necessarily support all new development indiscriminately. Their goals are not simply to disempower planning departments, eliminate red tape, and give developers free reign (Holleran, 2020). Many YIMBYs encourage housing development that caters to people from all walks of life and all income levels (Brown & Glanz, 2018; Schneider, 2018). A good portion of their suggestions—filling empty lots, creating transit-oriented development, making it easier for homeowners to add accessory units (i.e., “in-law” units)—are quite reasonable (Walker, 2018).

YIMBYs advocate for increases in housing supply at the regional level too—not only in San Francisco—so that suburban cities share some of the infrastructure burden and absorb some of the population (Holleran, 2020). Also, YIMBYs believe that housing proposals need to follow sound and sustainable planning principles if they are to be approved. An offshoot of YIMBYism is ‘Public Housing in My Backyard’ (PHIMBYism), whose followers advocate exclusively for public housing (Schneider, 2018). While YIMBYs have the support of developers and conservatives, as might be expected (Walker, 2018), many members are trained in urban planning, design, and architecture and believe that by supporting higher-density, more compact development they are following the profession’s consensus (Holleran, 2020).

On the positive side, YIMBYism is helping change planning conversations, and counterbalance NIMBYism (McCormick, 2017). The YIMBY Action Director states that “nobody would be talking about housing in the way they are if we hadn’t dramatically moved the discourse and that’s great” (Eskenazi, 2018). Partly owing to the push from YIMBY advocates, in 2016 the ‘100% Affordable Housing Bonus Ordinance’ was adopted in San Francisco. It provides density and height bonuses for developers in return for incorporating permanently affordable dwellings within a project for very low, or low, or moderate income households. If a residential development includes 20% affordable housing units on-site, then the development could be granted up to a 35% density bonus (San Francisco Planning, 2019).

Also in 2016, an amendment to the CEQA process was proposed, which sought to severely restrict NIMBYs’ capacity to legally challenge affordable housing proposals if these met all the local building codes. But only a diluted version of the law was approved in 2017, because as Wassmer and Wahid (2019: 344) explain, the proposed amendment.

faced resistance on multiple fronts, including: (a) environmental groups who saw it as undermining CEQA’s true intent, (b) localities who saw it as a threat to local control and land use, and (c) NIMBY groups who feared the loss of a tool they found effective at keeping affordable housing out of their neighborhoods.

Again in 2016, San Francisco’s Accessory Dwelling Unit Program became available citywide to accelerate the approval of secondary units, in-law units, or cottages added to existing houses. In 2019, there was substantial public support, but passage was not garnered for legislation requiring that California city governments allowed significantly more dense construction near transit hubs (Yglesias, 2018). While these bills have been lambasted for disregarding “the stark differences between heavily impacted central cities and exclusionary suburbs” (Walker, 2018:232), they suggest that at least a portion of the Golden State’s population has come to realize that housing density increases are paramount (though insufficient by themselves) to increase the supply of affordable housing (Gray, 2018a).

4 Data and methods

This study follows a qualitative approach to answer the research question (whether NIMBY attitudes continue to dominate planning hearings on housing development proposals, or whether other voices have gained momentum). Public participation is a cornerstone of the San Francisco planning process. Typically, all residents and owners of properties located within 150 feet of a proposed development and all registered neighborhood groups are notified of upcoming public meetings. In the case of large projects, developers are expected to meet with the local community prior to submitting a preliminary project assessment application, and once this has been reviewed by the City, they are expected to attend one or more public meetings. Based on San Francisco’s Inclusionary Housing Program, developers of projects with ten or more residential units are required to sell or rent a percentage of the units below market, a rate that is affordable to low and/or middle-income households (or pay an Affordable Housing Fee in lieu).

We review videos of Planning Commission meetings between 2018 and 2019 in the San Francisco City Hall in which housing development proposals were considered, and verbal input from members of the public was received. The San Francisco planning website (www.sfplanning.org/planning-commission) provides the following information on the role and composition of the Planning Commission:

The Planning Commission consists of seven commissioners appointed by the Mayor and the President of the Board of Supervisors. They hold weekly public hearings and maintain the San Francisco General Plan. The commission oversees the San Francisco Planning Department. … Members of the Planning Commission provide an invaluable service to the City: they advise the Mayor, Board of Supervisors and City departments on San Francisco’s long-range goals, policies and programs on a broad array of issues related to land use, transportation, and current planning. Additionally, the Commission has specific responsibility for the stewardship and maintenance of San Francisco’s General Plan. Planning Commissioners serve 4-year terms. Four of the seven Commissioners are appointed by the Mayor, three are appointed by the President of the Board of Supervisors.

The videos of the Planning Commission meetings are publicly accessible via the San Francisco Government TV website.Footnote 2 (The City and County of San Francisco, as many other public agencies in the US, routinely record public hearings and make them available online as part of their transparency programs.) A total of 20 videos were reviewed, which included more than 80 h of meeting footage. Meetings varied substantially in length, from one hour to nearly ten hours. A range of videos were purposively selected to provide coverage on a variety of topics including (1) affordable housing, (2) luxury developments, (3) re-zoning, (4) discretionary reviews, and (5) planning code amendments (Table 1). These meetings attracted more than 720 comments from approximately 550 participants. (It was impossible to determine the percentage of attendees who provided comments.) The time limit for each public speaker was usually 3 mList of San Francisco Planning Commission videos used in this study.

Table 1 List of San Francisco Planning Commission videos used in this study
Full size table

Each public commenter was tracked using an ID number. Estimates of the speaker’s age (in 10-year intervals, starting at 20), gender (male or female), race (white or non-white), and migrant status (US-born or migrant) were recorded along with any significant statements made. (Note that some people commented at multiple hearings.) Naturally, these estimates cannot be 100 percent accurate as they are based on people’s appearance and speech at meetings. For example, some foreign-born commenters may have migrated to the United States as children and therefore speak English with an American accent. Also, 5 percent of the population is of mixed race in San Francisco (US Census, 2020). However, these estimates provide information on the general trends in the public discourse. No information was available on the commenters’ place of residence.

The data was coded manually based on thematic analysis, in an iterative process which is common for qualitative studies. It is considered as appropriate when the research is focused people’s views, opinions, knowledge, experiences, or values—as in this case. We closely examined the data in order to identify common “themes,” in other words topics, ideas, and patterns of meaning that emerged repeatedly (Lune & Berg, 2017). While coding, we did not assess all aspects of the public participation process. Rather, we sought to discern whether instances of NIMBYism and/or anti-NIMBYism were present in public comments, and which group, if any, drove the public debate.

To contextualize the methodology: planning meetings and hearings usually happen during weekdays, and most working-class people impacted by gentrification are unable to attend them. (The data do not allow researchers to discern whether commenters are blue- or white-collar workers—unless a commenter volunteers this information.) The following section will show that, despite efforts to achieve inclusivity on part of San Francisco’s planning department, the playing field of public hearings is uneven and may be filled with various biases.Footnote 3

5 Findings and discussion

Our analysis, delineated below, reveals that, while anti-NIMBYism elements have a significant presence at development hearings in San Francisco, the presence of NIMBYs still dominates. It is impossible to determine with certainty that NIMBYism alone has exacerbated the housing crisis as we cannot directly measure the impact of NIMBYs on the votes of the Commissioners. However, we postulate that the persistence of NIMBY proponents does have an effect. Among the YIMBYs, a critical issue was whether or not a proposed development included an allocation for affordable units (inclusionary housing).

5.1 Limited participation and inclusiveness at planning meetings

Previous research has suggested that planning meetings are dominated by older retirees from the baby boom cohort (Einstein et al. 2019; Holleran, 2020). These are not necessarily “country club types”: in more progressive cities, NIMBYism is sometimes the province of “educated older hippie types” (Holleran, 2020:12). Our study has produced slightly different results. In San Francisco, 63 percent of commenters are over 40 but the largest percentage of commenters falls into the 40 to 60 age group—in other words, what may be considered as older members of Generation X rather than Baby Boomers (Fig. 2). It may be the case that, as the Baby Boom generation is aging, and Generation X is becoming established in the home ownership market, the latter is coming to represent the interests of homeowners.

Fig. 2
figure 2

Relative age of public commenters against San Francisco population, as estimated by the authors (One of the meetings (February 7, 2019) was excluded from this graph as it is an outlier. It was held at a local high school in the late afternoon, rather than the usual location in the city in the middle of the day. At this meeting, approximately 150 people commented which is significantly more than at any other meetings. The commenters also demonstrated a much larger age variation, with 65 percent estimated to be under 40. The source of data on total population is the United Sates Census Bureau)

Full size image

However, commenters tend to be older and “whiter” than the city average and do not mirror the local demographic distribution. Underrepresented in planning meetings are age groups 20–30, 30–40, and 70 + , while the white population is overrepresented at 65 percent, compared to 48 percent in the city overall. While younger participants tend to be equally split in terms of race, whites predominate by a large margin after age 40. The gender split of commenters is about 50/50, reflecting the city average. First-generation migrants (who speak English with a foreign accent) are underrepresented at 13 percent (compared to 35% in the city overall). (A translator is usually on site and used by 4% of commenters who cannot speak English). Hence, public participation at planning forums may not accurately depict the overarching attitudes of the general population.

Older commenters tend to have stronger views against development which can be associated with NIMBYism (Fig. 3). We find that a typical stance among this age group is wanting ‘to preserve the character of the neighborhood’. This is often repeated in their statements. However, a shift against NIMBYism is also evident—among younger commenters, renters in particular, many of whom lament a lack of affordable housing. A statement by the Planning Commission President summarizes this dynamic:

I am a member of generation X and I do think that there is a generational shift here between how the baby boomer generation and how millennials use land and the way that we use resources and… I think that it is something that we are not acknowledging here. (Meeting 1, female, non-white, 40–50 years).

Fig. 3
figure 3

Support for development proposals by age group, as estimated by the authors

Full size image

Commenters who explicitly state a reason for support or opposition of a proposal are represented in Figs. 4 and 5, respectively. Most commenters speaking in support of a proposal due to its inclusion of affordable housing are younger residents, in the 20–30 year age group. Similarly, younger people tend to oppose development that does not include affordable housing options. This could be attributed to the Millennial experience of being unable to afford to purchase a house, and even struggle to make rent payments. (This cohort has been dubbed “generation rent.”) But it could also be motivated by a more inclusive outlook among youth, and a lower tolerance for socio-economic inequality (Twenge, 2018). The comment below is representative of the views of the younger cohort of commenters:

One of the best solutions to address the homelessness crisis is by providing affordable housing opportunities…with every new market rate project that you approve, there is one less affordable housing development that we have to be able to place homeless families into. (Meeting 5, male, white, 20–30)

Fig. 4
figure 4

Reasons for support for development by age group, as estimated by the authors (Figures 4 and 5 include only those commenters who provided an overt justification of their opinion. There were many commenters who did not provide a reason for their support or opposition to a proposal; those were included in Fig. 3)

Full size image
Fig. 5
figure 5

Reasons for opposition to development by age group, as estimated by the authors

Full size image

There are also commenters who oppose new development generally but at the same time advocate for affordable housing. This is evidence of the PHIMBY phenomenon (support only for affordable housing) (Schneider, 2018).

Concern about affordable housing appears to dwindle as commenters’ age increases. A high proportion of older commenters (aged 50–60 years) oppose development based on other reasons such as access to a view or wanting to maintain their neighborhood character. The anti-development attitude of the older demographic could owe to the homeownership status of this group. Many may have purchased houses a long time ago, under much more favorable market conditions, and are personally unaffected by currently exorbitant rental or house prices; in fact, they may stand to profit from real estate bubbles. The comments below echo a general sentiment among older residents:

Just because everyone wants to live here, doesn’t mean everyone can live here (Meeting 1, male, white, 70+).

Just because people demand affordable housing doesn’t mean that we need to provide it (Meeting 6, female, white, 60–70).

Meanwhile, one regular commenter, who is a member of YIMBY Action, says:

The Homeowners Association came out here and told you their agenda pretty clearly. They said that their mission is to preserve the value of the neighborhood. The ‘fiduciary interest of the neighborhood’ is what they said. In other words, they’re here because they want high property values. And we maybe shouldn’t take that into account in our public policy making. The interests of a Homeowners Association are not the same as the interests of the people. We need more housing… (Meeting 18, female, white, 20–30)

But an anti-development stance among long-timers may also owe to resentment toward what is perceived as the “tech invasion” of the region. They feel that “their” city—the proud birthplace of the Black Panthers, the LGBTQ + movement, the Beat Generation, and other cultural landmarks—is being entirely transformed by Silicon Valley’s cataclysmic money and is selling its soul to venture capitalists and computer coders (see Bachrach & Christensen, 2014; McClelland, 2018; Solnit & Schwartzenberg, 2018; Walker, 2018). Manifestations of this sentiment are the longstanding disputes over the traffic congestion allegedly caused by tech company buses who transport their employees from San Francisco to Silicon Valley campuses (De Kosnik, 2014).

The conflict between NIMBY residents and others is clearly evident in all the planning meetings reviewed for this study. There are two distinct sides at every meeting, which at times become hostile and argumentative. However, the current public participation process does not appear to be effective at engaging all relevant members of the community equally for the purpose of achieving an optimal and equitable outcome. While many people do speak up freely at public meetings, commenters do not reflect the city’s demographic composition, described earlier. There are five to ten regular attendees at meeting who are well-versed on the topic at hand; they have often worked in, and retired from, the planning or property industry. The Commissioners welcome their input.

Limited representation is partly a result of the timing of meetings at City Hall. Meetings are scheduled during working hours and commenters lament that there is uncertainty over the length of meetings and the time when specific items will be considered. Some meetings last approximately an hour, while others continue for almost ten hours. While there is a public agenda, items are removed, continued, deferred, or taken out of order at the last minute and without notice. Therefore, participants cannot be sure if their item of interest will be tackled. This is inconvenient for the majority of working people and those with childcare duties. As one pro-development housing advocate states at a regular meeting:

They [working people] have a difficult time in this somewhat democratic process of a Thursday afternoon. ‘Can you take off work and ditch for an hour to wait in line for 25 min to speak to a group of people that will not actually be voting on anything.’ (Meeting 1, male, white, 20–30)

At one meeting, which exceptionally was hosted at a local school hall in the evening and offered on-site childcare, the number and diversity of public commenters increased dramatically. There were more people of color among the attendants and speakers, as well as people who work during the day.

In addition to inconvenient timing, the negative environment that prevails at public meetings is also not conducive to people wanting to attend and comment. Many of the meetings are volatile and can easily turn extremely acrimonious. (At some meetings much time was wasted when the Commissioners had to stop the proceedings and calm the crowd in order to continue.) There is also evidence of neighbors using public hearings to argue long-standing disputes or even blatantly slander one other. Accusations like the following are not uncommon:

[Commenter X] has a track record of making unfair and retaliatory allegations in an effort to boost his own personal net worth at the expense of the people around him. (Meeting 12, male, white, 40–50)

It appears that planning meetings provide participants with a platform in which to vent pent-up anger. However, studies have pointed out that people who are less assertive (for example, youth or women) or migrants with weaker English language skills may be intimidated by this atmosphere (see Listerborn, 2007; Ortiz Escalante & Gutiérrez Valdivia, 2015).

5.2 NIMBYism on display during public meetings

The data reflect a world where commenters proceed to recount stories incorrectly, quote false or misleading information, and/or justify their opinions through personal emotive experiences. This is evident because many assertions made by individual commenters are then contradicted or corrected by Commissioners, planning staff, or other public commenters. Many commenters do not directly speak in support or opposition of a development proposal being discussed, but rather recount personal experiences and emotions. Then other commenters “piggyback” off these ideas and comments. For example, commenters frequently mention the potential price decrease of their homes caused by new development. In response, the Commissioners regularly state that this does not appear to be the case and that research does not support the view that new housing development reduces the value of existing homes. The State Senator, Scott Weiner, has also publicly stated that:

These arguments don’t hold water. Indeed, in recent history, we’ve never come close to producing enough housing to allow anyone to argue that increasing housing supply doesn’t stabilize housing prices. (Weiner, 2015)

While Weiner is considered as a vocal YIMBY supporter, independent research conducted at the Institute of Governmental Studies at the University of California at Berkeley has concluded that “at the regional level, both market-rate and subsidized housing reduce development pressures, but subsidized housing has over double the impact [on affordability] of market-rate units” (Zuk & Chapple, 2016:1). At a meeting, a Commissioner clarifies:

I saw a lot of negative comments…somebody saying [that] the city is legalizing ‘homeless tent cities’. I’m not sure where this comes from… I would be very disappointed if there is, indeed, this type of communication in the public, to really push back on something which I believe is constructive, forward-looking and well within the city’s way of doing things. (Meeting 3, female, white, 60–70).

Yet, these opinions persist. Many of the commenters appear to have an overarching distrust towards all developers, believing that an entire occupational category is hostile, intractable, and solely driven by profit. This outlook hinders developers, including those who can potentially increase the affordable housing supply, and promotes NIMBYism. In some cases, meeting participants do call out those who are misinformed or seek to manipulate others:

A lot of MissionFootnote 4 residents resist new construction due to fear of gentrification and displacement, but that fear is largely based on anecdotal data that was put out by the project opposition…For the past 6 years, the opposition has relied on anecdotal data to make false arguments. I support this project because I believe in a need for housing for all. (Meeting 5, male, non-white, 20–30)

The comment on NIMBYism in more impoverished areas stemming from fear of gentrification is salient. Residents are concerned that any new housing units here will be seized up by higher income people, thus pushing up all prices in an area—not only housing but also food and other goods. Anti-gentrification activists in San Francisco are known for their stance that “YIMBYs should basically full-stop not advocate for housing in gentrifying communities” (Dougherty, 2020:229).

Discretionary Review (DR) cases are particularly polarizing during public meetings. NIMBYism is especially evident during those. By way of explanation, a DR takes place when the Planning Department or a member of the public requests the Planning Commission to exercise its discretionary power and review a proposal. A DR is meant to be used in “exceptional and extraordinary circumstances” as determined by neighbors of a proposed project (City and County of San Francisco, 2018:1):

Once the assigned planner determines the minimum standards are met and the project is approvable, the Department will mail a notice to residents and property owners within 150 feet of the subject property, as well as registered neighborhood organizations. The notification period provides neighbors with the opportunity to assess the project and determine whether the project creates or contains any exceptional and extraordinary circumstances.

The Planning Department makes a recommendation to the Planning Commission on whether the DR case should be approved, and the Commission tends to follow that recommendation. However, YIMBY Action (2017) points out that DR “is a costly process whereby frivolous objections can slow down affordable housing for months or years.” Academic research has found that discretion lowers public trust in the planning process and leads to discord (Kwok et al. 2018). The YIMBY advocate quoted above states at a meeting:

[DRs] pit neighbor against neighbor in this very destructive and hostile way that is sad and disappointing. … This is why planners quit. This is why the Planning Department is constantly losing qualified people: because they cannot handle this. This is the worst that our city has to offer. (Meeting 18, female, white, 20–30)

6 Conclusion

It is perhaps ironic that some of the most socially progressive American cities also feature some of most virulent forms of NIMBYism (Dougherty, 2020). While 1990s’ inhumane revanchinsm that blamed the poor for urban problems has abated in some cities, such as New York and Los Angeles (DeVerteuil, 2019), this study shows that NIMBYism (a residual of the revanchism) continues to dominate the discourse at public hearings on development proposals in San Francisco. As such, it is poised to overwhelm other voices and needs and consequently undermine social mix in San Francisco. Even as a major housing crisis has settled in, and additional homes are needed for low-and middle-income residents, not to mention thousands of homeless, opposition to new development is widespread. While alternative movements are gaining momentum, anti-NIMBY voices are hardly the most powerful in the city. Planning meetings appear to be dominated by older, white, homeowners who are more concerned about their property values and other personal hindrances than about the common good. Poorer, younger, and “ethnic” residents (particularly renters), while pro-development and pro-affordable housing, remain underrepresented. This issue is not simply generational, as proposed by YIMBY supporters: in the US context, class and race are central to the housing precariat (Holleran, 2020).

Public comment at planning meetings allows people to exercise their democratic right and freedom of speech. At the same time, planning meetings provide a forum in which residents can vent their frustrations, be they with neighbors, developers, or the tech industry that has appropriated the city. To the extent that Planning Commissioners base their votes on public comment, meetings provide an opportunity for NIMBY attitudes to impact planning decisions. This state of affairs in what was once a most radical and progressive city, supports an outcome unfavorable to the public need to improve housing affordability. It has undermined San Francisco’s famed diversity and is precluding social mix in its residential neighborhoods. Beyond NIMBYism, other factors that produce housing crises and preclude social mixing – speculative investing, financial excess, tax havens, and socio-economic inequality (McElroy & Szeto, 2017) cannot be addressed through planning processes or even housing legislation alone (Walker, 2018). They necessitate systemic changes in American society and polity.