Race After Technology

  • the tool should never possess the human

  • highly policed neighbourhoods carry a heavy sense of surveillance

  • big data and surveillance

  • naming as protection and assimilation

  • names as markers interacting with technologies, affects those who are ‘racially marked’

  • ‘objective’ technology likely has racial bias embedded

  • mariame Kaba ‘hope is a discipline, reality is something we create together, except that so few people have a say in the world in which they are forced to live’ […] ‘we can’t resign ourselves to this reality we have inherited’

  • names can open and shut doors, in employment etc

  • naming as branding

  • ‘a symptom of the larger neoliberal rationale that subsumes all other socio-political priorities to economic growth, competitive positioning and capital enhancement’

  • the stakes are high as people don’t often change their names, outside of queer communities

  • Anglicised names

  • African-American names as ‘comically made-up’, misspelled — that’s just how language happens

  • presumed blandness of White American culture

  • ‘the invisible center against which everything else is compared’

  • invisibility as a superpower, an immunity

  • escape responsibility for your role in unjust systems

  • names are social codes, they are not neutral, and often treated in predictable ways

  • names are concocted in cultural laboratories

  • the gap in employment is equivalent to 8 years of experience

  • new technologies that recreate existing oppressions behind a

  • ‘codes are both reflective and predictive, they have a past and a future’

  • codes act as narratives, telling us what to expect

  • Kathy O’Neill ‘racism is the most slovenly of predictive models. It is powered by haphazard data gathering, and spurious correlations reinforced by institutional inequities and polluted by confirmation bias’

  • a combination of racialised names and zip codes can flag someone as a potential threat

  • once flagged it can be difficult to lose association with that data

  • nicknames allow us to work around systems that intend to fix us

  • codes create a vast array of distortions and dangers

  • cut through industry hype to see how the desire for objectivity and progress (and capital) cause ‘technological fixes’ in social arenas

  • decode the promises of tech with reasonable skepticism

  • appearance of neutrality and benevolence

  • discriminatory designs that actively reinstate hierarchies

  • Kalil mohammed - precedent, earlier racial data revolution in the 19th century, racial statistics, pseudobiology etc

  • ‘datafication of injustice’

  • the language of progress is weaponised

  • new racial caste system under carceral systems

  • Jim Crowe started as folkloric but became academic shorthand for legalised racial segregation oppression and injustice in the US South 1890s - 1950s, New Jim Crowe as shift to ‘colorblind’ ideology

  • online search results associate black names with arrest records

  • James Baldwin - “The great force of history comes from the fact that we carry it within us, are unconsciously controlled by it in many ways, and history is literally present in all that we do”

  • digital caste system structured by existing inequities

  • the tech could often not exist without data produced through histories of oppression

  • tech sees cultural differences in all of its good and bad

  • instructions designed to fix problems

  • the US electoral college is an algorithm that inherently classes black lives as 3/5 of white lives

  • ‘culmination of particular tools, people and power systems that foreground one way of seeing’

  • tech designers encode judgements

  • racial bias is magnified but buried under ‘digital denial’

  • interference in the 2016 US election, blatantly bad actors and tech corporation coverups

  • ‘private industry decisions are public policy decisions in disguise’

  • influenced strongly by liberatarianism

  • ‘silicone valley entrepreneurs don’t mind the government regulating other industries but they prefer Washington to stay out of their own business’

  • their power isn’t meaningfully redistributed

  • ‘move fast and break things’ often breaks people and places in the process

  • data sharing by governments - sold as streamlining services but flags follow across multiple departments eg a bank lender having access to health file

  • Data fusion centers that coordinate data between local police, state police, intelligence agencies and private companies

  • we should stop calling ourselves users

  • allowing tech giants to represent our interests

  • more push lately to actively deconstruct networks of disinformation/‘Russian bots’

  • content moderation efforts often don’t actually address fundamental power imbalance and private decision-making

  • tech company as monarchy, and whether they have ruling over entities in other nations

  • consuming digital and physical real estate

  • consumers ‘voting’ for tech industry

  • data openness is the cornerstone of silicone valley

  • many tech insiders ban devices for their own kids

  • education tech is couched in access for all, but access goes both ways

  • algorithms too prominent, too many people listening to computer scientists, so ‘digital elites’ taking a high position in the hierarchy

  • no ‘personal right to refuse’ for most

  • ‘they are atomised and quantified, of having their personal uniqueness sold to them, one tailored experience after each other’

  • coded inquiry can be met with collective resistance

  • depersonalised personalisation

  • inherent sociality of learning

  • a liberatarian ethos that assumes we want to be left alone - hyperinvidualisation

  • tailoring and targeting

  • understanding race as a tool designed to stratify and sanctify social injustice as natural everyday life

  • white US Americans use race as a tool to denigrate the other

  • “the deadly demarcations that racial imagination brings to life”

  • White victimisation and false equivalences eg White lives matter

  • Baldwin - “White is a metaphor for power”

  • ethnically tailored niches that capitalise on calls for diversity

  • eg Netflix tailored recommendations with Black supporting cast, marketing adjusting their visibility

  • illusion of progress

  • using search histories and prior watching as a proxy for demographic stats

  • economic recognition not the same as political and social power

  • assumes people’s value hinges on spending money and spending attention

  • opting out of digital life can be criminalised - what are you trying to hide?

  • not to ‘diversify’ television but to normalise it - make it look normal, like the world actually does

  • ‘feel good differences’ like food and dress rather than systemic issues with education, housing, employment

  • pinkwashing

  • identities treated as immutable characteristics that can be strictly codified and sold

  • multicultural representation marketed as individualised experience quickly gets taken up by law enforcement

  • subtle forms of discrimination that give the impression of progress and neutrality

  • coded inequity makes it faster and easier to produce racist outcomes, regardless of intention

  • targeted and deliberate white supremacy still common, conduct coordinated harassment campaigns

  • gamification of hate by relying on individual reports

  • more traffic = more profit even if the traffic is violent clash

  • ‘digital mob’

  • ‘white supremacists love Twitter because Twitter loves them back’

  • white supremacists embed in seemingly benign content, dogwhistles

  • ‘slow death’ where subtle forms of inequity get a pass

  • ‘vast carceral apparatus that facilitates legal discrimination against those marked with a criminal record’

  • whilst Black people in theory have greater freedom of movement, in practise many are immobilised, a disproportionate amount

  • entire families and communities penalised by association

  • restrictions on where people can live, work and travel

  • more ‘objective’ measures — turning to technological fixes

  • new forms of obfuscation

  • virality allows anyone to publicise racist transgressions, cathartic ‘dragging’ as a collective ritual exposing everyday dangers.

  • video in particular positions the viewer as a witness whose judgement can have political repercussions for the perpetrator

  • hashtags and memes as a public square, collective punishment

  • by drawing our attention to obvious cases it might obscure the systematic failings, as does popular black media and celebrities gaining attention

  • tokenism - “the Black face of technologies” - eg Will.i.am?

  • as an art form hiphop pushes technological boundaries eg DJing beatboxing sampling

  • Apple valuing and devaluing blackness at the same time - again valuing the aesthetic and arts of Black people without the actual experience

  • there is no way to create something without some intention and intended user in mind

  • in tech development there’s pressure to conform to the prejudices of the world

  • ethnically tailored marketing reinforces racial faultlines - opportunistically oppressive

  • computational approaches ‘necessary costcutting’ - outsourcing burdens of gatekeepers as’bias minimisation ’ which insources tech bias instead

  • consequential decisions

  • opportunity to ‘manage’ social lives

  • shaping citizen subjects who prioritise efficiency over equity

  • what does it mean to be human - philosophical crisis

  • its capital exceeds sovereignty

  • redefinition of human identity autonomy core constitutional rights and democratic principles

  • our notion of humanity is fragmented - nonhumans and not quite humans

  • genres of humanity tell different stories about past present future

  • posthuman living beyond the body - assume we have all had the chance to be human

  • black people already live in the future - eg experimented on, surveillance, legislative test subjects

  • ‘slippery slope between effective marketing and efficient racism’

  • examples of algorithmic bias - redirecting real estate ads away from low-income (ethnically diverse) neighborhoods, higher paying jobs advertised to white men, standardised testing charged at higher rates in high Asian populations

  • ‘black box’ useful metaphor here, obfuscation of tech function

  • how the law is used to aggressively prortect commercial secrecy whilst ignoring our right to privacy

  • black box as recording device and mysterious object. anti-black box

  • race-neutral laws serve white supremacy

  • protect poor whites from aid requirements by reducing requirements in high unemployment areas in rural areas. urban becomes black, rural becomes white

  • not explicitly invoking race, but invoking things that practically mean race

  • race as technology that separates, stratifies and sanctifies injustice experienced by racialised groups

  • construct symbolic devices to structure society

  • race varies across time and place and is codified in law and culture

  • racial logics permeate tech development

  • racism is a means to reconcile the contradictions of history’s dehumanisation of racialised people

  • freedom isn’t free but not everyone pays the price

  • luddism remembered as out-of-touch when in reality it was a social protest against innovation leaving the working class behind

  • “to break the machine was in a sense to break the conversion of one’s self into a machine for the accumulating wealth of another”

  • oppose automation, called a luddite

  • impact of rollouts and transformation

  • ‘demanding a more careful and democratic approach to technology’

  • imagined futures

  • ‘all the work without the workers’

  • electronic sweatshops - Amazon workers and micro-monitoring

  • ‘lower costs and greater efficiency’ - disposability of labour that enables innovation

  • ‘opportunity to work’

  • racing after technology - efficiency demands

  • deterministic worldview - worldview shown of technology - engine of human progress or decline

  • techno-determinism

  • progress for whoever is in charge - but anything that simply allows

  • lag period of society catching up with its laws and norms

  • but this assumes technology as neutral, developed outside socio-political context and shaped by our inputs

  • ‘technology is society and society cannot be understood or interpreted without its technological tools’

  • ‘social norms ideologies and practises are a constitual part of the technological design’

  • digital divide and unequal access along predictable lines (race, class, gender, etc)

  • ‘a focus on technophobia and technological illiteracy downplays the structural barriers to access and also ignores the many forms of tech engagement and innovation that people of color engage in’

  • african continent expected to leapfrog innovation as not held by clunky older tech infrastructure

  • ‘racialised boundary constructed between low and high tech’

  • ‘different minorities have different functions in the cultural landscape of digital technologies’ - eg ‘asian as solution and black as problem’

  • a colorblind internet assumes text-only internet

  • blackness as the ‘anti-avatar’

  • race impacting who has access to devices

  • surveillance pervasive in black lives - ongoing from slavery ‘overseer’

  • precondition for the fabrication of technology

  • consider racism alongside other forms of domination

  • inevitable, automatic and ingrained but not permanent

  • ‘an invitation to refuse the illusion of inevitability in which technologies of race come wrapped and to hotwire more habitable forms of social organisation in the process’

  • ‘high theory vs pop culture, academic vs activist, evidence vs anecdote’

  • thin description - reading surfaces such a screens and skin - being racialised is to be p

  • John l Jackson - ‘thin description is about how we all travel, through the thicket of time and space, about the way both of those trajectories might be constructively thinned, theorised concretised or dislodged in service to questions about how we relate to each other in a digital age’

  • thick description passing itself as a surpassing how people see themselves

  • thinness as a more elastic approach to knowledge production

  • antitode to digital disconnection

  • tracing links between individual and institutional, mundane and spectuacular, desirable and deadly

  • respecting particular kinds of boundaries

  • thin description doesn’t fall for the trap of aiming for complete and total knowledge

  • information veiled for the sake of story

  • new jim codes for all-knowing extractive work

  • thin description as acceptance of fragility

  • we’re likely to miss the next form of veiled racism

  • glitches as technological canary in the coalmine

  • weights of visibility and being watched but not seen

  • technologies that work for fairness but fairness is defined narrowly and reproduces and creates new cleavages

  • methods of advocating for justice-oriented design practises

  • ‘technical fixes to social problems’

1 - Engineering

  • Beauty-contest judged by AI - awful

  • ‘beauty is in the trained eye of the algorithm’

  • strictly classified ethnicity and used label-matched images

  • combined preferences of everyone’s data involved

  • using AI as preventative public health initiative - darker people implicitly coded as unhealthy and unfit - eugenics !

  • valuable health info can come from just portrait images

  • in deep learning, depth refers to layers of abstraction, learning complicated concepts from simple concepts

  • autotagging or preferences most common uses

  • learning as a human would - is there only one theory of mind and who’s mind are we modelling?

  • “the allure of objectivity without public accountability”

  • “enforcement of racial hierarchies with real consequences, embodied in robots”

  • “akin to humans and at times superior”

  • outsourcing decisions that should be in democratic oversight

  • private companies developing the AI, acting as public entities without oversight

  • technocracy - ‘governing without a mandate’

  • Virginia eubanks predictive analytics by US social welfare agencies, automated decisions aim to mitigate fraud by depersonalising the process” historically we’ve only got better in social work is by extra-personalising

  • hurting the most vulnerable

  • discrimination is displaced

  • issues of intentionality and visibility

  • I tinker therefore I am

  • robots portrayed as humanoids

  • blurring the line between maker and made

  • uncanny valley

  • a robot is any machine that can perform a task directed by humans or programmed to operate automatically

  • “the most advanced are smart machines designed to learn from and adapt to their environments, created to become independent of their makers”

  • “an anthropology of robots” - historically been a way to talk about dehumanisation whilst not talking about racialisation

  • from czech word compulsory service, servitude, hardship

  • threat of annihilation, war machines and human domination over other humans

  • people are the first robots

  • master-slave disk - virtual hierarchies, signifiers inherently borne from human social dynamics

  • “by 1965 we’ll all have personal slaves again” robot slaves - “we” as readers mean white readers, we the people with inalienable rights

  • how we imagine our relation to robots gives a frame for thinking about race as technology.

  • disposability

  • ‘throwbots’ and bomb disposal bots - ‘own the real estates with their eyes rather than their bodies’

  • intertwined history of machines and slaves

  • we imagine being enslaved by robots but the tech labour force is already deeply unequal

  • outsourced labour - Disposable Women of Global Capital

  • different digital class - entrepreneurs are the new economic men

  • large naturally occuring datasets rife with bias are the base material for robots to learn

  • “if the training data is produced by a racist society, it won’t matter who is on the team, but the people who are affected should also be in the team”

  • as becoming more human, more racist, unless we actively restructure the techno-social environment

  • “machines cannot themselves be racist, even equipped with artificial intelligence they have neither brains nor intentions” - but that isn’t what racism is, it’s the wider systemic affect

  • lack of intent isn’t important, they are perpetuating the environment

  • “racism flourishes well beyond hate filled hearts”

  • plugging in anonymous numbers without malice is widely impactful

  • legal codes, financial practices and medical care embed racism despite no central brain

  • “clothed in the rhetoric of public safety”

  • “aiming to do good can very well coexist with forms of malice and neglect. In fact, a do-gooding ethos often serves as a moral cover for harmful decisions.”

  • racist technology does not imply every individual involved is racist

  • complex power dynamics

  • norms and structures

  • “by focusing mainly on individuals’ identities and overlooking the norms and structures of the tech industry, many diversity initiatives offer little more than cosmetic change”

  • “learn to speak the coded language of their human parents, not only programmers but all of us online who contribute to naturally occuring datasets on which AI learn”

  • “individual [racial] identity offers no surefire insulation from the prevailing ideologies”

  • you can’t just deploy it and leave it alone, it will decay over time

  • “uncomfortable birthing stage of artificial intelligence”

  • value judgements in 1s and 0s

  • if data scientists so indeed treat their robots like children […] then I propose a race-conscious approach to parenting Artifical life - one that does not feign colour blindness

Automating antiblackness

  • stereotype and default settings
  • print stereo plate used to make copies
  • “image perpetuated without change”
  • etymology of this term urges sustained investigation of interconnections of technical and social systems
  • codified racial stereotypes only one angle - eg credit scores for trustworthiness and automatic judgements
  • sifting and sorting large datasets that we can’t
  • ‘objective calculations and scores’
  • focus less on intended uses and more on their actions
  • computational agents
  • Simians, Cyborgs and Women - The Reinvention of Nature - blurred boundary between organism and machine. Myth and tool mutually constitute each other. Technologies as frozen moments that allow us to observe otherwise fluid social structures. Instruments that enforce meanings and help construct the social world
  • black mirror
  • words that signal undue privilege are not legible
  • power is if anything relational
  • if there’s an underside there’s an upside
  • if we don’t have words to discuss these power dynamics then
  • it’s possible to change computer systems so they don’t distort our view of social systems
  • we can do it for simple but we must also do it for complex
  • anti blackness precursor to all surveillance
  • consider the stakes of the simpler systems - see the similarities, hygiene and racialised language - darker skin tones associated with dirtiness and low health - from marketing to politics - ‘racial hygiene’
  • near-infared technology built with skin reflection, not suitable for dark skin
  • making black people invisible when they want to be seen Vs in other situations when visible when they want to be invisible, in surveillance
  • black mirror, nosedive, service workers and quality
  • social credit programmes are actually used in life already
  • racial discrimination is not limited to one country or system but border crossing
  • China’s social credit system started in 2014, mandatory enrollment from 2020, using systems from private companies, proprietary financial algorithms on work , criminal activity, leisure activity, purchases
  • investigates and so shapes behaviours
  • Muslim Chinese people forced to download terrorist tracking app. Trumps goal to register all Muslims
  • social ‘misdeeds’ bar wider financial activity, jobs etc
  • “ensure the bad people don’t have a place to go”
  • higher score gives extra privileges
  • ‘objective’ ranking of ‘merit’
  • ‘likes’ on social media favour marketers and disadvantage protestors and dissenters
  • “automated social credit systems make a broader principle of merit based systems clear. Scores assess a person’s ability to conform to established definitions of good behaviour and values sociality rather than measuring any intrinsic quality. More importantly, the ideological commitments of dominant groups typically determine what gets awarded credit in the first place, automating social reproduction”
  • “multiple axes of domination typically converge in a single code’
  • ‘idle” citizens, abelist logic of unproductive,
  • “the conflation of economic productivity and upright citizenship is ubiquitous among many societies’”
  • parenthood via purchases
  • reproductive excess, assumptions about sex and morality
  • eugenics of race related reproduction
  • geography of social worth - your postcode is really impactful
  • ‘digital birth control’
  • ‘how abstract quantification gets embodied’
  • ‘readable on the surface’ thinking qualities should be obvious on the surface
  • fictional experiences that mirror experience of oppressed people
  • ‘we’ might be next - whiteness ‘threatened’
  • ‘lack of intention to harm is not a viable alibi’
  • is the ‘glitch’ systemic
  • false, spurious, nterruption or irregularity, slippery
  • Malcolm X as Malcolm 10
  • database design […] is an exercise in world building
  • “a” normative process in which programmer’s are in a position to reproduce their worldviews, a process that all to often reproduces the technology of race”
  • displacement and gentrification via mapping
  • “reflect and reproduce racialised commands that instruct people where they belong in the larger social order”
  • those who frame the problems - reading Roman numerals and parsing them correctly was an ‘innovation’
  • periodic glitches
  • “narrow investment in technical innovation necessarily displaces a broader set of social interests. This is more than a glitch. It is a form of exclusion and subordination built into the way in which priorities are established and solutions defined in the tech industry”
  • innovation as smaller and more neutral than progress, less implication of social improvement
  • racism doesn’t require a high level of visibility
  • “between fleeting and durable, microinteractions and macrostructures, individual hate and institutional indifference”
  • glitches as ‘a signal of how the system operates’
  • feelings are a form of evidence, community testimonial is data
  • people affected KNOW what these technologies are doing to harm
  • them
  • recitivism risk scores - remarkably unreliable in forecasting violent crimes, and incredibly disproportionately racialised
  • ‘the weather’ as the total climate
  • forecasting involves anti-blackness as their datapoints are affected by climate of anti-blackness eg ongoing surveillance, lack of housing and education, etc
  • ‘rhe weathermen are also the ones who make it rain’
  • if prediction matches the crime rate it is still unjust, but the measures are still used as judgement of fairness
  • model used from earthquake aftershocks
  • temporary crime zone - expecting crime and finding crime, is passing through the crime zone considered probably cause, most probably
  • Deja vu as subconscious pattern recognition, useful for uncovering systemic inequalities, “seemingly trivial technical glitches and meaningful design decisions”
  • ‘matrix of domination’, potential sites of resistance
  • media ritual of ‘dragging’, maybe welcome distraction from more insidious forms
  • “stigmatising individuals is not much of a deterrent and rarely addresses all that gives them license and durability”
  • ‘bad apple’ metaphor applies here, as individuals aren’t standalone but are poisoning the whole ‘orchard’
  • glitches are not transient but durable signals of inequalities
  • institutuonal racism coined in 67
  • “Institutional racism has been maintained deliberately by the power structure and through indifference inertia and lack of courage on the part of the white masses as well as petty officials … The line between purposeful suppression and indifference blurs” - Hamilton and Ture 1967 p. 38
  • cyber-racism ‘white supremacy has entered the digital era’
  • focus on top-down forces in systemic racism but those subordinated often have their own forms of agency, dissent and resistance- “structural racism encounters self-reflective action” - ” the US system of racial hegemony where despotism and democracy coexist in seemingly permanent conflict”
  • researchers tend to concentrate on how the internet perpetuates or mediates racial prejudice at the individual level rather than analyse how racism shapes infrastructure and design
  • urban planning, impossible to hold large gatherings
  • hostile public space
  • connections in the building of physical and digital worlds
  • legal, social and building codes
  • structures, highways, low hanging underpasses, prevention of buses passing- prioritisation of suburbanisation and middle class mobility over public transport
  • physical construction is central to exclusion, continually
  • “the way we engineer the material world reflects and reinforces, or could also be used to subvert, social hierarchies”
  • scalped trucks on unusually low bridges
  • “tools of social exclusion are enormous guaranteed to impact only those who are explicitly targeted to be disadvantaged through discriminatory design”
  • “algorithms that are constantly updated on the basis of human behaviour, and are learning and replicating the technology of race, expressed in the many different associations the users make”
  • computers can be racist because racism isn’t about intent
  • signals
  • “the pornography industry has billions of dollars to throw at companies in order to optimise content advertising can not continue to be the primary driving of online content” - Noble (algorithms of oppression?)
  • “premature optimisation is the route of all evil” - Donald Knut
  • data mining, scoring and predictive software will ‘amplify historical and institutional associations’
  • anything that understands language well enough to produce it will learn historical connections and culture
  • AI is given agency in our society
  • “I think my blackness is interfering with the computer’s ability to follow me”
  • exposure in cameras, exposure socially
  • ignoring some humans is not indifferent tech!
  • “tech advancement posed as a solution conjures a prior racial regime” - separate fountains due to sensor tech failure
  • “indifference to blackness can be profitable within the logic of racial capitalism until the social costs become too high to maintain”
  • “multiply exposed” - surveillance and risk
  • “visual technologies and racial taxonomies fashion each other” capture visual evidence of difference and empire, “allure of objectivity”, staged photographs
  • positive bias towards lighter skin tones built into colour photography - colour balance and digital techniques
  • “the act of viewing something, or someone, may put the object of vision at risk”
  • “scopic vulnerability” - ‘black skin, white masks’
  • “threat of exposure and being misread”
  • “visibility of a person is visibility of a race” - external quote
  • “detection and recognition are not the same”
  • ableism and beauty Mia Mingus “moving towards the ugly - a politic beyond desirability” - how “the generational affects of global capitalism, genocide, violence, oppression and trauma, settle into our bodies”
  • “magnificence of a body that shakes, spills out, takes up space, needs help”
  • multiple exposure and ghost images
  • “liberatory forms of scopic resistance and recoding”
  • standardised exposure cards from industry standard exposure cards of a white woman - a choice.
  • multiple skin tones in one frame - technical fixes not catered to until product images had the same issues
  • Polaroid flash boost used to more accurately capture images for tracking books that control movements