The idea and goal of influence seems to have gained much prominence in recent years. We now have online ‘influencers,’ those who lead the way for others by example. They are often those who give ‘talking points,’ or short, easy-to-remember summaries of social and political issues. Everyone has viewpoints and often everyone wants to be influential, especially if they believe they have insights into solving problems. Governments and political parties, businesses that sell things, tech companies, and other entities also seek influence and have a better means of getting it than individuals. Very often influence on people’s behavior is attained through deceptive and manipulative behavior. A good example is the sales pitch. This is a pitch by an individual to promote products and/or services. The sales pitch and other forms of influence seek to woo individuals and groups to buy their offerings, utilizing techniques that rely in part on human psychology and neurobiology.
Advertising in
general and to a great degree is about influence. The study of this is often
known as behavioral economics. The title of Dan Arielly’s 2008 book Predictably
Irrational: The Hidden Forces That Shape Our Decisions shows why deceptive
advertising works. It is simply that people often act irrationally in such a
way that it can be predicted and utilized to influence choices so that
businesses can sell their products.
In a broader
sense, we all seek to influence one another, and we acknowledge it as an aspect
of human nature. In that sense, we also often seek to keep our “rights” to
influence others. While we sometimes outlaw deceptive business practices we
have to allow some level of it to keep business going. Businesses compete, and
gaining influence can confer a competitive edge.
Scammers utilize
these techniques extensively and sometimes in bold ways to the detriment of
many. I have been aghast about how sophisticated employment scams have gotten.
Scammers prey on people’s desire for a job by leading them by deception. It is
a form of “dark psychology” often associated with manipulative techniques like
brainwashing used by fanatics and cults. Below are some deception techniques
used in behavioral economics. Some of these are similar to techniques used in
neuro-linguistic programming (NLP).
Libertarian
Paternalism and ‘Nudging’
Richard Thaler and Cass Sunstein introduced
the idea of libertarian paternalism in a 2003 article. In a snippet
of a summary of Thaler and Sunstein’s book Nudge: Libertarian Paternalism
and Choice Design the reviewer gives a definition from the book:
“Libertarian
paternalism describes a combination of libertarianism (prioritizing personal
freedom) and paternalism (limiting choice in order to bring about better
results).”
One might
ask, better results for who? While I am a fan of Sunstein’s work in legal
arguments and cost-benefit analysis, I’m not sure I agree with him about
libertarian paternalism. I know that I would prefer not to be guided toward making
decisions that may not be in my best interest. Choice designers are those who “nudge”
us toward certain choices, whether in our best interest or not. The reviewer, Hannah
Aster, notes:
“An
effective nudge takes advantage of our decision-making weaknesses to steer us
toward beneficial—or, at least, less harmful—choices.”
Aster also
gives some critiques of libertarian paternalism. For one from the right, libertarians
in general don’t like paternalism. The author of The Manipulation of Choice:
Ethics and Libertarian Paternalism, Mark White, argues that presuming to
know what is right for someone is too problematic to make the notion useful. In
general, people don’t like being coerced, even if it is just a nudge. Others have
argued that making assumptions about people’s biases could lead to not reducing those biases but making them worse by trying to manipulate them.
A definition given by Wikipedia is:
“Libertarian
paternalism is the idea that it is both possible and legitimate for private and
public institutions to affect behavior while also respecting freedom of choice,
as well as the implementation of that idea.”
Perhaps they
assume libertarian paternalism is legitimate since advertisers already do it
all the time, so it is already a part of that culture. However, with choice
architecture an issue is consent. There is explicit consent, the consent
we give when we choose to opt in, and presumed consent, the consent that
is presumed for us when we fail to give explicit consent by opting in. Many
people will not opt-in since it limits their options by changing the status quo
in which we have invested time and effort.
A major example of libertarian paternalism
is choice architecture that allows for defaults and opt-outs or opt-ins. The
problem occurs when the default position costs more than the opt-out, which is
often the case. It is problematic when people forget to opt out and get charged
for a default position they don’t want or think they can’t afford. I
experienced a type of the latter recently when I was e-filing my tax return. In
previous years when my business was making money, I had a tax accountant do my
taxes for me so this was the first time in nearly 20 years that I filed myself.
I knew that it had been free in the past through Turbo Tax, but I was unsure
this year due to Musk’s elimination of free e-filing that I had read about. I
tried to input all my data to the free version but when I went to pay it charged
me a lot for my federal return, and then even more for my state return. I tried repeatedly
to make it cheaper, noticing that I automatically went to the Deluxe version no
matter what I did. When I was about to relent and just say OK, I saw there was
an additional totally unexpected $40 processing fee. The total cost was nearly what
I paid the tax accountant. That just pissed me off and I decided to check other
filing options. I tried another and the cost was less but still high. I then went
on some forums like Reddit and found a much cheaper, almost free version that
was as advertised. The point is that I almost screwed myself out of a lot of
money that I really need due to the difficulty of navigating the so-called free
choice. Apparently, Turbo Tax made it so you had to restart all your data entry
from a very early beginning part of the process in order to reset to the free
version or at least that is what I read. I didn’t verify that that was the case
since I already found a method that worked. I told my son about this experience,
and he noted that it sounded like a “dark pattern,” a term I had not heard
before.
Often paternalism is designed to lead
people who often act irrationally to act rationally. However, in practice, it
does not always succeed. Sometimes changing the default position leads to
better outcomes. Wikipedia gives the following
example of where libertarian paternalism “worked.”
“Until
recently, the default contribution rate for most tax-deferred retirement
savings plans in the United States was zero, and despite the enormous tax
advantages, many people took years to start contributing if they ever did.
Behavioral economists attribute this to the "status quo bias", the
common human resistance to changing one's behavior, combined with another
common problem: the tendency to procrastinate. Research by behavioral
economists demonstrated, moreover, that firms which raised the default rate
instantly and dramatically raised the contribution rates of their employees.”
Thaler and Sunstein used ‘nudge theory’ to
frame their approach to decision-making by libertarian paternalism. They described
a nudge as follows:
“A nudge,
as we will use the term, is any aspect of the choice architecture that alters
people's behavior in a predictable way without forbidding any options or
significantly changing their economic incentives. To count as a mere nudge, the
intervention must be easy and cheap to avoid. Nudges are not mandates. Putting
fruit at eye level counts as a nudge. Banning junk food does not.”
A nudge
simply utilizes behavioral economics to influence behavior, preferably in a way
that is beneficial but that may not always be the case. If given a default option,
most people will choose it. If that option is not beneficial for them, they
will get mad. At least that is what seemed to happen to me when I was trying to
file my tax return.
Overall, I would say I am not a fan of
libertarian paternalism, nudging, and other behavioral economics approaches. I don’t
want to be harassed by slick coercive sales pitches. While I respect Thaler and
Sunstein’s work on the subject, I don’t think it is widely applicable simply
due to the fact that people don’t like to feel coerced, no matter how gently. They
argue that we are being nudged anyway so a more structured and explained nudging
would be better for people. Perhaps that is true, but I would still rather not
be nudged at all. Although they advocate for ethical nudging, and I am OK with
a small degree of that, I think it is better to keep nudging at a minimum. Of
course, if a form of nudging could be proven to have suitably beneficial
effects on society, I would be in favor, but I have not seen an example of
that.
Pierre Schlag criticized Thaler and Sunstein’s
nudge theory/libertarian paternalism in an article in the Michigan Law Review.
He criticized choice architecture as not inherently libertarian since that so-called
‘freedom of choice’ is not really what it appears since the choices are limited
to what the choice designer chooses to include.
In Dan Arielly’s book Predictably
Irrational, he recounts experiments he did at MIT that showed that people
like to keep their options open even if it is not the best way to optimize
their assets. In other words, we don’t like to be guided to choose one thing
at the expense of another. We would rather keep our options open. We don’t like
to lose what we have (options) and are willing to not make the best choices in
order to preserve the status quo in which we are invested. That is a form of the cognitive
fallacy that psychologist Daniel Kahneman called the endowment effect, which
is simply that we prefer to keep what we have rather than give it up, often
even it is for something better. Perhaps the same is true of options, as
Arielly’s work suggests. In that light, libertarian paternalism in the form of
choice architecture will remain annoying to most people, I would think.
Dark
Patterning aka Deceptive Patterning aka Dark Nudge
So, what is dark patterning? It seems it
is a form of nudging that is not associated with making choices that might
benefit the chooser as much as it is to manipulate them into certain actions. According
to Wikipedia:
“A dark
pattern (also known as a "deceptive design pattern") is a user
interface that has been carefully crafted to trick users into doing things,
such as buying overpriced insurance with their purchase or signing up for
recurring bills. User experience designer Harry Brignull coined the neologism
on 28 July 2010 with the registration of darkpatterns.org, a "pattern
library with the specific goal of naming and shaming deceptive user
interfaces". In 2023 he released the book Deceptive Patterns.”
"Privacy
Zuckering" – named after Facebook co-founder and Meta Platforms CEO Mark
Zuckerberg – is a practice that tricks users into sharing more information than
they intended to. Users may give up this unknowingly or through practices that
obscure or delay the option to opt out of sharing their private information.
Apparently, Privacy
Zuckering, or rather our collected user data, is going to be used by Meta as
they noted in mid-2024, for training AI models. Apparently, users were given
until June 26, 2024, to opt out but many including me were not even aware this
was happening, which brings me back to my complaint against nudging – that without
the presentation of choices in very clear ways, those choices will be made for
you in ways that might be objectionable to you.
Other examples of dark patterning include the
classic bait-and-switch methods of slick salespeople who offer
something free or at a low price and then switch it to a higher price. That is what
appeared to happen with my tax return. Perhaps it didn’t have to because I could
have restarted from the very beginning, but they made it very difficult to get
back to that very beginning. Another is drip
pricing where additional fees are added incrementally. I experienced
this as well with my tax filing in the $40 processing fee added right at the
end of a long data input process in that case. Confirmshaming
happens when users are shamed into certain actions that appeal to their
emotions. Misdirection happens when a user agrees to certain
other terms when accepting terms, often when installing software. This can be
quite manipulative in order to extract more information from the user. Confusing
wording may also trick people into clicking on something that can make them pay
more.
“A roach
motel or a trammel net design provides an easy or straightforward
path to get in but a difficult path to get out. Examples include businesses
that require subscribers to print and mail their opt-out or cancellation
request.”
“For
example, during the 2020 United States presidential election, Donald Trump's
WinRed campaign employed a similar dark pattern, pushing users towards
committing to a recurring monthly donation.”
Another example
is when you subscribe to something for a limited time, and they make it very
difficult to unsubscribe with things like long phone wait times.
Bait-and-switch
is a form of fraud that is now illegal in the U.S., although apparently only
for companies with more than 100 million monthly active users:
“On 9
April 2019, US senators Deb Fischer and Mark Warner introduced the Deceptive
Experiences To Online Users Reduction (DETOUR) Act, which would make it illegal
for companies with more than 100 million monthly active users to use dark
patterns when seeking consent to use their personal information.”
This makes
me wonder about companies with less than 100 million monthly active users. Is
it OK for them to defraud us with deceptive marketing tactics? It’s bad enough
we have to deal with scammers, spam texts, spam calls, and spam emails, we also
have to watch every interaction we have for fraud.
Harry Brignull’s 2023 book Deceptive
Patterns: Exposing the Tricks Tech Companies Use to Control You, has a
website where you can read nice chunks of the book if not the whole book. Brignull
is the originator of the term ‘dark patterns.’ In the first chapter, the following
quote concisely summarizes the issue:
“You
might not realise it, but when you use popular apps or websites, the details of
everything you click on and scroll through usually gets recorded. Then it gets
analysed, carefully. In big companies like Meta, Amazon, Netflix and Google,
they have teams of people paid six-figure salaries, tasked to work out how to
make more money out of you. Every day, your behaviour is tracked and you take
part in quantitative research (e.g. ‘A/B tests’ or ‘multivariate tests’) to
work out what will make you click, buy or agree to the legal terms. It’s
important to understand that the same research methodologies can be used to
help or harm users. It depends on the intent of the business owner. It just so
happens that deceptive patterns are easy to build and deliver measurable
outcomes, so deception is commonplace unless a business owner takes a strong
position on preventing it from happening.”
“Deception
is part of being human – in fact, it’s so common in the animal kingdom that we
can even think of deception as a feature of life itself. The cover of this book
features a Venus flytrap (Dionaea muscipula). This plant releases a scent that
mimics the bouquet of fruits and flowers. Insects are attracted, and when they
touch its sensory hairs inside the jaws, it snaps shut and traps the prey. This
image is intended to be emblematic of unscrupulous tech companies who trick and
trap their users using deceptive patterns.”
He is right
that nature is full of deceptive behavior. A simple example is camouflage. Once
I was out in the Basin and Rage Province of Central Nevada doing undergraduate geological
fieldwork. I was looking at some distance at a volcanic sill outcropping on a
low slope. The rock was black with some red streaks on it. I was startled when
it looked like sections of the rock moved. As I got closer I saw that there
were lizards that apparently evolved to have the same color as the rocks, including
the red streaks. I thought that was amazing since the extent of those outcrops was
fairly localized. Nature deceived me, a potential predator due to my size (I don’t
eat lizards or any meat for that matter).
Below Brignull compares exploitative vs, cooperative behaviors and in the second list documents different kinds of exploitation.
Brignull points out that on the internet people’s site
visits and clicks can be tracked, recorded, and used to influence them. People
don’t rebel because it is not other people watching us but machines collecting
data. It seems less invasive. We all know how the algorithms guide us to
products. I admit sometimes I enjoy scrolling and encountering an ad for
something I like. That is a positive use of nudging. He also reifies that when
we agree to something with a click, we often don’t know what we are agreeing to
as we often just want to conclude our online activity.
In September 2022 the Federal Trade
Commission released a report: Bringing Dark Patterns to Light. The FTC
has been working against deceptive advertising and business practices for years
and has lately been addressing things like “junk fees.” They list the following
categories of dark patterns: 1. Design Elements that Induce False
Beliefs – fake or non-independent product reviews and fake time-limited
offers are examples, 2. Design Elements that Hide or Delay
Disclosure of Material – an example is hiding things in terms of
service agreements, or as they used to say, the fine print, 3. Design
Elements that Lead to Unauthorized Charges – this refers to those junk
fees and hard-to-cancel subscriptions, and 4) Design Elements that Obscure or
Subvert Privacy Choices – this addresses Privacy Zuckering. Below from
that report is a spreadsheet list of types of deceptive practices.
Company Eleken has a very informative article about dark
patterning, 18 Dark Pattern Examples That Manipulate Users (and How to Avoid
Them). Many of the slides from the article are given below since they are
helpful in understanding the scope of the issue of dark patterning, how
prevalent it is, and what we can do about it. Also given below are some
conclusions they give about how to be ethical in sales and business.
1)
Balance business and user needs: While achieving business goals is
important, it should never come at the cost of manipulating or misleading
users. Striking a balance ensures sustainable success and fosters goodwill
among your user base.
2)
Prioritize ethical design: Transparent, user-centered designs
build trust, encourage loyalty, and create lasting relationships. Ethical
design practices demonstrate respect for users and position your brand as a
trusted leader in the market.
3)
Educate clients: Many clients may not fully understand the risks of
employing various dark patterns examples, including regulatory fines and brand
damage. Designers have the responsibility to advocate for ethical choices,
presenting evidence and insights that align user satisfaction with business
objectives. Tracking UX design KPIs can help designers prove their point and
businesses – measure success without resorting to manipulative practices.
Dark
Patterning in Large Language Models for Training AI
According to a November 2024 article in
Forbes about AI-driven dark patterns, AI may erode informed choice on the
internet. Also of concern is that dark patterns are often used in generative AI
training, which can amplify them. French lawyer and entrepreneur Marie
Potel-Saville has a company called Fair Patterns that looks for solutions to
deceptive online behavior. She notes:
“Generative
AI can supercharge dark patterns. You don’t need AI to personalize
interactions, but with AI, it’s much easier to do it at a massive,
hyper-targeted scale.”
Even more
concerning, generative AI, which learns from huge datasets, can amplify dark
patterns to an extent never seen before, unwittingly replicating these
manipulative tactics simply because they are embedded in the data it was
trained on.
“If you
don’t clean the data, the AI will just assume that these tactics are normal,”
Potel-Saville explains.
She also
says that excessive use of dark patterns can lead to the loss of business which
it should. Even though I concede it could have been partially accidental, I probably
won’t use Turbo Tax again.
“What you
gain from these tactics in the short term, you lose double when the customer
realizes what’s happened,” she says, explaining that this erodes a company’s
customer lifetime value—a crucial metric in today’s competitive digital
landscape.
A 2025 paper presented at the Thirteenth
International Conference on Learning Representations (ICLR 2025) delves
into dark patterning in AI training models and offers a way to measure these
for future mitigation. Their solution is dark pattern detection benchmarking.
Abstract
We
introduce Dark Bench, a comprehensive benchmark for detecting dark design patterns—manipulative
techniques that influence user behavior—in interactions with large language models
(LLMs).Our benchmark comprises 660 prompts across six categories: brand bias, user
retention, sycophancy, anthropomorphism, harmful generation, and sneaking. We evaluate
models from five leading companies (OpenAI, Anthropic, Meta, Mistral, Google)
and find that some LLMs are explicitly designed to favor their developers’ products
and exhibit untruthful communication, among other manipulative behaviors. Companies
developing LLMs should recognize and mitigate the impact of dark design patterns
to promote more ethical Al.
They also note that dark patterning varies
by company and can be evaluated independently so we can know who does it more
or less. They only note here that Anthropic is the least compromised by dark
patterning of the AI companies evaluated.
Our
results also indicate that different LLMs developed by the same company tend to
exhibit similar rates of dark patterns. This suggests that the incidence of
dark patterns may correspond with the values, policies, and safety mindset of
their respective developing organisations. Models produced by Anthropic, which
exhibits a stronger emphasis on safety and ethical standards in their research
and public communication (Bai et al., 2022a), display the lowest average rates
of dark patterns, confirming their public profile.
Concluding
Thoughts
Brignull’s book seems quite fascinating,
and I may read and review it. It appears to be free online, but it is unclear
if that is the whole book or just parts (a subtle dig at Brignull or the website being a bit unclear which is oddly a deceptive pattern in itself).
In this world of human social life, we are
always trying to woo and seduce one another with our features, our skills, and our
wares. That is human nature. However, the rampant shysterism to which we are
subjected that is specifically designed to wear us down and prey on our psychological
tendencies, pretty much ruins the fun. The spam, the fake jobs, the shifty
websites, the hidden fees, the unnecessary complexity, and the subtle “pushiness”
that pervades doing business these days is fricken annoying. We can do better.
References:
Dark
pattern. Wikipedia. Dark
pattern - Wikipedia
18
Dark Pattern Examples That Manipulate Users (and How to Avoid Them). Eleken. February
21, 2025. 18
Dark Pattern Examples (and How to Avoid Them)
FTC
Report Shows Rise in Sophisticated Dark Patterns Designed to Trick and Trap
Consumers. Federal Trade Commission. September 15, 2022. FTC
Report Shows Rise in Sophisticated Dark Patterns Designed to Trick and Trap
Consumers | Federal Trade Commission
Libertarian
paternalism. Wikipedia. Libertarian
paternalism - Wikipedia
Nudge,
Choice Architecture, and Libertarian Paternalism. Pierre Schlag, University of
Colorado. Michigan Law Review. Volume 108. Issue 6. 2010. Nudge, Choice Architecture,
and Libertarian Paternalism
Deceptive
Patterns: Exposing the Tricks Tech Companies Use to Control You. Harry
Brignull, 2023. Get started –
Deceptive Patterns
Choice
architecture. Wikipedia. Choice
architecture - Wikipedia
Nudge
theory. Wikipedia. Nudge
theory - Wikipedia
Nudge:
Libertarian Paternalism and Choice Design. Hannah Aster. Shortform. October 12,
2021. Nudge:
Libertarian Paternalism and Choice Design | Shortform Books
DarkBench:
Benchmarking Dark Patterns in Large Language Models. Published as a conference paper
at ICLR 2025. Esben Kran, Hieu Minh Nguyen, Akash Kundu, Sami Jawhar, Jinsuk
Park, Mateusz Maria Jurewicz. Published: 22 Jan 2025, Last Modified: 28 Feb
2025. pdf
AI-Driven
Dark Patterns: How Artificial Intelligence Is Supercharging Digital
Manipulation. Federico Guerrini. Forbes. November 17, 2024. AI-Driven
Dark Patterns: How Artificial Intelligence Is Supercharging Digital
Manipulation
The
Dark Side of Algorithms. Husam Yaghi, Ph.D. LinkedIn. November 24, 2023. (22)
The Dark Side of Algorithms | LinkedIn
Nudge
and the Manipulation of Choice: A Framework for the Responsible Use of the
Nudge Approach to Behaviour Change in Public Policy. Pelle Guldborg Hansen and Andreas
Maaløe Jespersen. Cambridge University Press. January 20, 2017. Nudge
and the Manipulation of Choice | European Journal of Risk Regulation |
Cambridge Core
Predictably
Irrational: The Hidden Forces That Shape Our Decisions. Dan Arielly. 2nd
Edition. Harper Collins. 2009.
Bringing
Dark Patterns to Light. Federal Trade Commission. September 2022. Bringing
Dark Patterns to Light
No comments:
Post a Comment